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EXECUTIVE  SUMMARY 

DOD-STD-2167A  is  a  U.S.  milium'  standard  establishing  requirements  for  sollware 
development.  Much  of  the  operational  software  currently  being  developed  lor  the  Australian 
Department  of  Defence  is  being  developed  in  accordance  with  this  standard;  its  u.sc  is  likely  to 
be  mandatory  in  almost  all  future  operational  systems. 

The  transition  to  this  new  standard  has  not  been  .straightforward.  There  have  been  numerous 
criticisms  about  DOD-STD-2167A  particularly  wiili  regard  to  the  amount  of  documentation 
required  tnd  the  effort  needed  to  produce  it.  Doubts  have  also  been  raised  as  to  the  actual 
value  of  such  documentation  to  the  customer  or  developer.  These  problems  are  not  unique  to 
2167A,  however.  Similar  problems  w-erc  experienced  with  earlier  .standards,  but  2167A  is 
affecting  more  projects  and  developers  and  thus  is  perceived  as  a  larger  problem. 

This  report  is  the  culmination  of  a  study  into  the  u.sc  of  2I67A  in  Australian  software 
development  projects.  .A  prior  report  addrc.sscd  the  problems  experienced  in  the  use  of  the 
standard  in  this  country. 

The  study  was  planned  to  be  as  open  as  possible.  In  addition  to  a  comprehensive  survey  ol 
2167A  p<ilic\  and  usage  in  the  early  stages,  drafts  of  both  reports  were  widely  distributed  to  all 
interested  parlies  atid  workshops  were  held  in  Canberra  and  Adelaide  to  discuss  the  findings. 

DOD-STD-2 167A  is  used  in  projects  because  it  is  the  prescribed  standard  for  most  Defence 
.software  developments  in  Australia.  In  some  cases  customers  outside  Defence  specify  it 
because  it  is  the  only  well  known  comprehensive  standard  available. 

It  is  not  simple  to  use  -  developers  with  less  structured  development  environments  find  it 
particularly  difficult  to  understand.  It  also  leans  heavily  on  other  military  standards  which  must 
also  be  understood  if  it  is  to  be  used  effectively.  In  some  cases  there  are  conllicts  between 
different  standards  which  make  the  task  more  difficult. 

DOD-STD-2 167A  is  neither  perfect  nor  complete.  It  contains  contradictions  and  omissions 
which  must  be  rectified  by  tailoring  the  standard.  Successful  use  of  this  standard  is  critically 
dependent  on  the  quality  of  the  tailoring.  Tailoring  must  be  applied  for  all  projccLs,  and 
whenever  possible,  tailoring  should  be  carried  out  as  a  joint  effort  between  customer  and 
developer. 

Most  problems  with  2167A  stem  from  the  following: 

•  The  standard  is  pot'rly  understood  by  either  the  customer  or  developer  or  b<ith. 

•  The  standard  is  olicn  inadei|uately  tailored  to  meet  the  needs  ol  the  project. 

•  The  developer  does  not  have  a  systematic  development  process. 

•  There  is  insufficient  meaningful  communication  between  the  customer  and  developer. 
Not  surprisingly,  the  following  principles  for  using  2i67A  arc  recommended: 

•  Provide  appropriate  education  and  training  for  all  staff  involved  in  the  use  of  21fi7A. 


DOD-STD-2 lb7A  must  be  tailored. 
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•  Special  consideration  needs  to  he  given  lo  enhancing  the  communication  and 
cooperation  hclwecn  customer  and  developer, 

•  The  development  methodology  selected  for  the  project  must  match  the  requirements 
of  2I67A  (appropriately  tailored)  and  must  he  adhered  to. 

I  here  is  no  "silver  bullet"  for  tailoring.  Tools  and  guidelines  arc  available  but  they  arc  no 
substitute  for  knowledge  and  experience.  They  merely  assist  in  the  process  of  tailoring,  but  not 
iti  making  the  hard  dceisiotis  which  will  arise.  Tailoring  mu.st  also  consider  the  family  of 
statidards  to  be  used,  not  DOD-STD-21ft7A  in  isolation.  Guidelines  in  2167A  and  its  tailoring 
handbook  also  restrict  the  scope  of  tailoritig  to  less  than  what  is  necessary  in  most 
circumstances,  reducing  the  advantages  that  tailoring  can  achieve.  There  arc  numerous 
occasions  when  the  tailoring  should  exceed  that  suggested  by  the  guidelines. 

It  appears  to  be  popular  to  eritici.se  216'7A  lor  its  dependence  on  the  "waterfall"  model  of 
software  development  -  to  claim  that  it  is  outdated  and  inappropriate  for  "modem"  development 
mclhodologics.  The  authors  suggest  th;ii  many  such  criticisms  arc  based  on  an  inadcc]uatc 
understanding  of  2t67A,  often  coupled  with  an  unwillingness  to  impose  systematic  control  on 
the  development  process.  An  inability  or  unwillingness  to  apply  appropriate  tailoring  is  also  a 
barrier  to  adapting  2iri7A  to  different  models. 

.Specilic  recommendations  arc  made  lor  reducing  the  amount  of  documentation,  streamlining 
reviews  and  audits  and  identifying  the  relationship  of  the  standard  to  testing  and  conliguralion 
management. 

rite  study  has  also  identified  areas  lor  further  research  in  the  development  and  acquisition  of 
Delence  .software  in  Australia.  Thc.se  include  the  assessment  of  the  capability  of  developers  and 
customers  to  participate  in  2167A  projects,  problems  in  the  preparation  of  RFTs  and  tenders  lor 
soltware  intensive  projects,  the  use  of  electronic  documentation,  the  establishment  of  2167A 
resource  repositories  and  the  application  of  vcrillcation  and  validation  (V&V)  to  Defence 
projects. 

I  bis  report  offers  advice  and  recommendations  which  enhance  the  understanding  and  use  of 
D( )D-.STD-2 167A,  and  which  arc  already  helping  to  improve  the  development  of  software  for 
Delence  projects.  The  most  important  recommendation,  however,  is  that  both  customers  and 
developers  must  dedicate  more  cflort  to  the  education  of  their  staff  -  DOD-STD-2  lfi7A  is  a 
viandard  which  does  not  lorgive  ignor;mce  or  amateurism. 


Copies  of  this  dorumenl  on  nui^neiie  medio  ore  ovoiloMe  from  the  oiilhors  on  request. 
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I  INTRODUCTION 


1.1  Purpose 

nOD  STD-2 167A  is  a  U  S.  miliiary  standard  establishing  requirements  lor  sollware 
development.  Much  of  the  operational  .software  currently  being  developed  (or  the  Australian 
Department  of  Defence  is  being  developed  in  accordance  with  this  standard;  its  use  is  likcls  to 
be  mandatory  in  almost  all  future  operational  systems. 

I  hc  transition  to  this  new  standard  has  not  been  straightforward.  There  have  been  numerous 
criticisms  ab<iut  DOD-STD-2 1 67A  (and  its  precursor  DOD'STD-2 1 67).  particularly  with  regard 
to  the  amount  of  documentation  required  and  the  effort  needed  to  produce  it.  Doubts  have  also 
been  raised  as  to  the  tictual  value  of  such  documentation  to  the  customer  or  developer. 

This  report  is  the  culmination  of  a  study  into  the  use  of  DDD-S'I  D  2167A  in  Australian 
software  development  projects.  It  makes  recommendations  lor  the  use  and  tailoring  ol  the 
standard. 

.\s  the  initial  phase  of  this  study  the  authors  conducted  a  survey  ol  2167.A  policy  anu  I'saee  in 
software  development  pixiiects  |(i.AR<>l  |.  .Although  the  study  was  primarily  aimed  at  defence 
projects,  there  are  several  non-delence  a|iplicalions  of  the  standard  in  Auslralia.  both  in  internal 
devclopmcnt.^  and  in  cotninercial  applicatit'its.  These  were  also  considered  in  the  sur\’c\'. 

A  drafi  of  this  repiui  wtis  circulated  to  interested  pailies  for  comment  and  was  the  subjecl  ol 
workshops  held  in  Canberra  and  Adehude  in  April/May  1602.  This  n''>al  report  includes 
feedback  from  these  acti\  ilies 

The  opinions  expressed  in  this  pajX'i  are  those  of  the  authors  ;md  do  not  represent  the  policy  or 
official  stand[xiini  ol  DS'IO  or  the  Departtneni  of  Defence, 

1 .2  Scope 

This  report  provides  disciissKin  about  and  recommeiKlalions  for  the  use  .iinl  Itiiloiing  in 
DOD-SrD-2l67A  projects.  Much  ol  the  material  is  based  on  the  assumption  that  the  reader  is 
generally  conversant  with  the  rerjuirements  of  2167A,  and  that  the  soltware  development  occurs 
as  the  result  of  a  contract  between  att  organisationally  distinct  custt'mer  and  developer. 

Initially  it  was  assumed  that  the  majorcau.se  of  dilTiculties  itt  the  use  ol  2167A  in  Australia  was 
in  the  complexity  of  the  t;iiloring  activitv  and  it  was  therefore  intended  that  this  study  would 
concentrate  mainly  on  the  tailoring  of  2167A.  The  survey  indictited  lhat  tailoring  was  otily  [>atl 
of  the  problem  anil  the  scope  ol  the  study  was  broadened  to  encompass  the  more  general  issues 
of  2I67A  use  Consequcnilv.  less  ellort  has  been  directed  towards  providing  detailed  guidelines 
for  tailoring. 

Similarly,  this  report  docs  tiot  address  other  asficcts  of  .soltwstre  dcvciopmeni  projects  which 
some  readers  may  regard  as  critical.  One  exantpic  is  the  relationship  ol  2I67A  to  quality 
standards  such  as  DOD-STD  2168,  AS  ,^66^  and  AS  .WOl. 

1.3  Nnmcnclaliire 

The  terms  "customer"  and  "developer"  arc  used  in  this  paper  to  indicate  those  responsible  for 
software  system  procurement  and  development  resju’ctively.  In  some  cases  the  tenns  are  used 
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u>  indiculc  individuals  in  the  customer  anG  development  teams  rather  than  the  organisation  that 
il)e\  represent.  The  tenns  "users"  and  "mainiainers"  are  used  to  indicate  those  'aIio  will  use  ihe 
soltware  (the  operators  (dr  example)  and  those  who  \vill  maintain  it  in  the  cusinmer's 
organisation. 

1.4  Organisation  of  this  report 

Section  2  examines  the  purprase  (or  2167A,  some  of  its  deficiencies  and  experiences  wiiti  its  use 
in  .Australia. 

Section  recommends  a  basis  for  the  use  of  the  standard  -  advice  to  customers  and  developers 
III  preparing  to  use  the  standard, 

SectuMi  4  discusses  the  general  principles  of  tailoring  2167A  and  related  standards. 

Sections  5  and  (^  address  the  general  requirements  of  2167A  and  the  allocation  and  pariiiionine 
ol  ilie  basic  soltware  clentcnis.  Computer  Software  Configuration  Items  (CSCIs).  cc'inponcnts 
T'SCs)  and  units  (CSl's). 

Section  7  addresses  the  development  process  and  development  methods,  their  relationsliip  with 
.’IfiT.A,  and  possible  changes  to  the  process,  methods  or  the  standard  that  may  he  necessary. 

Sections  S  and  examine  the  documentation  issues,  firstly  in  a  general  sense,  then  lor 
mdividual  ilocuments. 

1  inalU.  Sections  10  and  1 1  discuss  the  necessity  of  lurther  work  and  the  coitclusioi  s  oi  the 
study 
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2  l)()l)-Sri)-2167A  -  THKORY  AND  PRA(  TK  K 

2.1  The  reasons  for  usinj;  DOD-SI  n-2l67A 

DOD  S  rO-2l67A  is  usually  used  in  projects  not  by  choice  but  because  it  is  the  prescribed 
standard  lor  solivvare  development  I'hc  developer  uses  it  because  it  is  sju'caficd  in  the  contitiei; 
the  cusiotner  sfx'cincs  it  either  because  it  is  ilic  customer's  prescribed  policy  to  do  so.  or 
Ix'cause  there  tire  no  other  reasonable  altenuiiivcs  There  are  of  course  many  standards  which 
address  the  issues  td  soltware  development,  but  none  which  covers  the  .ame  breadth  with  ihe 
sarne  le\els  ol  prescription  as  this  standard 

One  ;idv;mtage  ol  usinc  2lri7A  is  that  it  is  p;irl  ol  a  (very  large)  standards  family  which  should 
cuaranlee  its  consistency  with  other  standards  It  meets  this  objective  reasonably  well,  although 
not  without  some  [iroblems  iSee  secti  ii  4  4)  Requirements  not  delined  in  21b7,A  arc  generally 
[iror  ided  in  other  lehiled  military  standards,  reducing  the  likelihcHid  ol  omissions  and  conllicis 

It  also  provides  a  consistent  basis  ior  management  ol  a  soltware  derelopmeni  project  by  the 
eiisionier  As  Mil  HDBK  2H7  desenbes. 

It  establishes  standeod  terminology,  provides  a  standard  set  ol  deliverables,  reviews 
and  audits  to  choose  horn,  and  defines  a  standard  set  of  soltware  management 
practices  that  may  be  imposed 

Visibility  ol  the  rievelopmeni  is  .issurerl  not  only  by  the  reviews  and  tiudiis.  but  tilso  by  the 
dettiiled  requirements  lor  the  roniuii  and  the  content  ol  dte  delivered  documentation,  the 
system  ol  reviews  also  provides  increasing  visibility  as  the  design  becomes  more  detailed, 
.allowing  the  etiOiviier  to  assess  tx)lh  the  ellcct  of  design  choices  on  the  opv'iational  rerjuirement 
and  tl'.e  developer's  ability  to  complete  the  devcloirment  succcsslully 

I'inttllv.  it  irnixrses  stamUirds  lor  the  development  process  that  are  arguably  higher  than  those 
that  m.inv  cirntractors  might  employ  il  not  obliged  otherwise.  In  this  way  the  customer  tries  to 
guarantee  a  reasonable  level  ol  (Quality  lor  the  product. 

Ihese  reasirns  are  important  in  understanding  how  2lb7A  should  be  used  and  how,  iir  il.  it 
should  K'  changed  (tailored)  lor  a  particular  project 

2.2  Relaliofislrip  lo  olhcr  standards 

1401)  STn-2lb7,A  h;is  been  designed  to  be  part  ol  a  large  lamily  ol  integrtited  military 
stand.irds,  and  it  assumes  that  sevenil  ol  these  siamlards  will  also  tx'  spc'cilied  when  using 
21b7A.  providing  an  intcgriiicd  iramework  lor  software  development.  If  one  or  more  ol  these 
st.indards  are  not  sfx’cilicd.  21b7A  must  be  tailored  both  to  remove  the  references  to  theiti  ;ind 
to  comfX'nsate  lor  the  fact  that  their  reqtiiremenis  am  no  longer  included.  The  relevant 
standards  are: 
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el  led  of  the  tailoring  on  the  remainder  of  the  standard,  resulting  ir,  conflicts  wiili  other  parts  of 
2I67A  and  with  other  standards. 

Perhaps  more  serious  is  the  authors’  experience  that,  whether  2167A  is  tailored  or  not, 
contractors  rarely  meet  tlieir  full  obligations  with  regard  to  the  standard.  In  these  cases  the 
shortfalls  are  often  overlooked  by  the  customer’s  monitoring  team  or  not  enforced  when 
delected. 

There  is  afso  evidence  that  both  customers  and  developers  concentrate  their  attention  on  the 
documentation  deliverables  at  the  expense  of  the  development  requirements. 

Developers’  criticisms  of  the  "inappropriateness"  of  2167A  documentation  requirements  often 
stem  from  the  lack  of  a  systematic  approach  to  development.  DOD-STD-2167A  requires  a 
i^ystcmatic  development  method  and  its  documentation  requirements  reflect  this.  Problems  arise 
when  developers  attempt  to  document  a  poorly  structured  existing  design  using  the  2I67A 
formats  (MCG9()].  More  generally,  2167A  specifies  activities  and  provides  a  strict  framework 
for  their  documentation.  If  the  activities  are  not  performed,  or  are  not  carried  out  in  the 
reijuired  mantier  or  order,  the  documentation  proce.ss  is  difficult  and  the  resulting  documentation 
is  likels  to  lx*  of  low’  value 

It  is  interesting  to  note  that  while  developers  are  concerned  about  the  amount  of  documentation 
requited.  m;m\  customers  Ix’lieve  that  developers  tend  to  provide  loo  much  data  in  requirements 
and  design  documents,  while  omitting  critical  information.  It  appears  that  some  developers 
anticipate  that  a  customer  is  less  likely  to  reject  a  thick  document,  perhaps  based  on  previous 
ex(X'nenccs,  The  autfiors  fiave  obsen'cd  some  evidence  of  truth  in  this  belief. 

Mati\  projects  have  seemingly  been  burdened  by  a  lack  of  understanding  and  experience  with 
?.  lb7A  on  the  part  of  the  developer,  the  customer,  or  both.  Developer  inexperience  has  resulted 
in  difficulties  iti  developing  software  and  its  associated  documentation  to  the  required  standard. 
Customer  inexperience  has  resulted  in  inadecjuatc  and  inconsistent  tailoring,  and  an  inability  to 
adequtitely  monitor  the  development  and  review  delivered  documentation.  The  inevitable  result 
of  inexp<.'ncncc  is  that  either  or  both  of  the  parties  are  unable  to  identify  limitations  in  the 
development  process  or  areas  of  potential  risk. 

One  of  ihe  side  effects  of  developer  and  customer  inexperience  is  that  some  developers  do  not 
assess  the  full  costs  of  using  2iri7A  in  tenders  for  software  projects,  cither  through  lack  of 
understanding  of  llic  standard,  or  on  the  assumption  that  the  customer  will  not  enforce  what  arc 
in  fact  contractually  binding  requirements.  Not  only  is  this  unfair  to  developers  who  include 
Ihe  full  cost  of  2I67A  development  in  their  tenders,  but  it  also  adds  risk  to  Ihc  project  -  cither 
in  the  quality  of  the  product  if  the  full  requirements  arc  not  met.  or  in  the  developer  being 
obliged  to  meet  requirements  for  which  he  has  not  budgeted, 

2.6  Omissions  in  2I67A 

This  section  addresses  perceived  omissions  in  2I67A  and  is  mainly  concerned  with  the 
documentation  of  commonly  needed  information.  Thc.se  omi.ssions  should  normally  be  rectified 
by  cither  tailoring  existing  a’quirements  or  by  the  specification  of  additional  requirements  in  the 
SOW  (Statement  of  Work)  or  CDRi.  (Contract  Data  Requirements  List). 

2.6.1  System  design 

A  notable  omission  from  21b7A  is  Ihc  lack  of  a  requirement  for  an  overview  of  the  software 
system  design,  mapping  the  requirements  to  the  design  and  illustrating  how  the  functions  of  tire 
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system  arc  met  in  icmis  of  execution  and  data  How  (at  a  relatively  high  level)  (MARHX, 

SI’RH9|.  Such  information  is  extremety  n^dul  in  understanding  how  a  complex  system  works 
and  for  tracing  the  source  of  defects  during  maintenance.  To  some  extent  this  information  is 
included  in  the  Software  Design  Document  (SDD)  in  paragraph  .3.1  wiiii  regard  to  "stales  and 
modes "  but  it  is  inadequate  in  its  level  of  detail  and  description. 

Tor  a  system  of  several  interacting  CSCIs  there  is  no  obvious  place  for  such  a  description,  apan 
from  die  Systcm/Scgmeni  Design  Document  (SSDD)  which  is  at  loo  high  a  level  and  is 
delivered  too  early  in  the  project  In  this  ease  one  or  more  additional  documents  arc  needed, 
possibly  called  "System  Architecture  and  Design  Description",  coordinating  the  design  of  llic 
(\SCIs  The  authors  see  such  a  document  being  developed  incrcmcntaliy  in  a  manner  similar  to 
SDDs.  with  preliminary  infonnation  being  provided  at  the  Preliminary  Dc.sign  Review  (PDR). 

2.6.2  Documental  ion  overv  iew  and  glossary 

In  larger  projects,  an  overview  of  the  interaction  between  the  different  specifications  and  other 
documents  is  required,  to  allow  specialists  and  casual  project  staff  to  find  specific  information. 

A  project-wide  glossary  of  abbreviations  and  definitions  is  also  often  essential,  both  to  assist 
newcomers  and  to  maintain  a  consistent  nomcnc'-turc  throughout  the  project. 

2.6.3  I'ser  interface  design 

One  difficulty  that  .several  developers  have  faced  w'illi  2167A  is  its  lack  of  guidance  with 
respect  to  the  documentation  of  the  design  of  the  .sy.stcm’s  user  interfaces  fUIs)  |MEY89|.  The 
most  obvious  place  for  this  information  is  in  the  Interface  Dc.sign  Document  (IDD)  which  docs 
not.  however,  provide  appropriate  formats  for  its  definition.  The  Software  User's  Manual 
iSUM)  is  intended  eventually  as  a  manual  for  the  sy.stcm  and  will  be  derived  from  the  Ul 
design.  However  it  is  developed  far  too  late  in  the  procc.ss.  and  in  most  projects  would  not 
contain  all  the  design  information  needed. 

Ul  design  should  occur  early  in  the  development  process  and  be  reviewed  at  PDR  |  l.S21B|. 
Where  there  is  a  complex  Ul  its  design  should  be  specified  in  a  separate  document  (the  '"User 
Interface  Design  Document  "  is  an  apt  name):  if  the  Ul  is  simple  it  may  be  included  in  a 
tailored  IDD.  Mll.-STD-793.‘iA’s  requirements  for  its  "Users  Manual"  and  "End  User  Manual " 
mav  offer  assistance  in  determining  the  contents  of  such  a  specification  1793.SA!. 

2.6.4  (,'ustomer  interaction  in  requirements  refinement  and  design  choices 

In  many  projects  the  refinement  of  requirements  and  detailed  design  can  lead  to 
implementations  that  the  customer  regards  as  unsatisfactory.  This  is  particularly  likely  in  areas 
such  as  the  definition  of  the  user  interface  and  in  the  specification  of  detailed  performance 
(such  as  response  times).  More  importantly,  although  the  implementation  may  be  unacceptable, 
it  is  often  either  compliant  with  the  higher  level  requirements  or  the  judgement  of  compliance  is 
a  subjective  i.ssue.  While  it  might  be  cKiimed  that  such  a  situation  is  a  result  of  ptxirly 
specified  requirements,  this  will  frequently  not  be  the  case.  Customers  arc  encouraged  to  avoid 
detail  in  requirements  that  might  inhibit  the  design  (see  also  Section  9.2).  The  penalty  for  lack 
of  detail  is  a  development  resulting  in  an  unacceptable  design. 

It  is  obvious  that,  regardless  of  whether  the  design  is  compliant  or  not.  the  cost  of  rectification 
will  be  minimised  if  the  deficiency  is  detected  early.  The  review  process  required  by  2167A 
and  152 IB  is  insufficiently  responsive  to  .solve  this  problem,  and  is  likely  to  rc.sull  in  either 
additional  costs  borne  by  the  customer  or  developer  or  both,  or  the  deficiencies  not  being 
rectified. 


ERL-0637-RE 


In  most  projects  the  likelihood  of  such  a  situation  arising  can  be  predicted  and  catered  for  in  the 
development  process,  particularly  if  it  is  identified  as  a  risk  area  (which  it  is). 

DOD-STD-2167A  addresses  this  problem  only  broadly,  requiring  that  the  developer  identify 
amas  ot  "potential  technical,  cost  or  schedule  risks"  12167A  4.1.41.  The  SOW  should  include 
the  requirement  for  the  developer  to  additionally  identify  areas  of  potential  disagreement  or 
subjective  interpretation  and  to  propo.se  procedures  to  rc.solvc  thc.se  as  early  in  the  development 
process  as  possible.  These  might  include  for  example  user  interface  prototyping  or  the  use  of 
an  incremental  development  procc.ss. 

2.6.5  Software  representation  and  generation 

The  design  of  each  CSCl  will  often  result  in  specific  decisions  being  made  with  regard  to  the 
representation  of  the  software  (such  as  source  file  partitioning  and  naming)  and  the  environment 
in  which  the  software  is  to  be  generated,  including  the  u.sc  of  standard  libraries,  compiler 
directives  and  options  for  the  use  of  the  compiler  and  linker.  Where  these  decisions  arc  made 
as  part  of  the  detailed  design  process,  and  arc  therefore  relevant  to  the  coding,  integration  and 
testing  activities,  they  should  be  documented  as  pan  of  the  design. 

This  information  should  be  documented  in  the  SDD  for  each  CSCl.  CSC  or  CSU  as  applicable. 

2.6.6  Traceability  requirements 

DOD-STD-2167A  specifics  that  requirements  arc  to  be  traceable  from  the  high  level 
specification  (eg  SSS)  to  CSCls,  CSCs  and  CSUs,  and  from  the  CSUs  to  the  SRS  and  IRS. 

The  DlDs,  however,  require  this  information  in  a  different  form,  providing  traceability: 

•  From  the  SSDD  to  the  SSS  (in  the  SSDD) 

•  From  the  SRS  to  the  SSS.  CIDS  or  PIDS  (in  the  SRS) 

•  From  the  SSS,  CIDS  or  PIDS  to  the  SRS  (in  the  SRS) 

•  From  CSUs  to  the  SRS  and  IRS  (in  the  SDD). 

This  provides  traceability  from  CSUs  to  the  high  level  specification  in  an  indirect  form  (through 
the  SRS).  To  ensure  that  the  high  level  rcquiremenl.s  arc  met,  it  is  more  u.seful  to  have  a 
traceability  matrix  from  the  high  level  speeifieation  directly  to  the  CSCls,  CSCs  and  CSUs  (and 
to  the  IDD  if  necessary).  For  larger  projects  this  would  require  a  separate  document  u.scd  solely 
for  showing  traccabMity.  Where  automated  traceability  tools  arc  u.scd  this  should  entail  little 
extra  effort. 


3  BASIS  FOR  USING  2167A 

This  section  provides  observations  and  recommendations  for  using  DOD-STD-2167A. 

The  prime  recommendation  (which  might  appear  to  be  obvious)  is  that  both  customer  and 
developer  mu.sl  have  a  detailed  understanding  of,  and  preferably  experience  in,  the  requirements 
and  purpose  of  DOD-STD-2I67A  and,  when  specified,  the  related  military  standards. 

.T1  Education  and  training 

To  achieve  a  thorough  understanding  of  the  purpose  and  requirements  of  DOD-STD-2167A  and 
related  military  spccillcaiions  is  noi  an  ea.sy  ta.sk  and  requires  considerable  education  and 
training.  Tertiary  computing  qualifications  arc  desirable  but  by  themselves  arc  not  enough, 
since  graduates  are  not  adequately  prcpaied  in  these  areas.  Experience  in  real  software 
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engineering  is  critical,  especially  exposure  to  working  with  standards.  For  a  successful  project 
both  customers  and  developers  must  have  the  appropriate  levels  of  knowledge  and  experience 
IMCGQO.  MEY89,  SPR90!. 

Ideally,  all  development  staff  should  have  several  year’s  experience  in  software  engineering 
(preferably  in  the  same  application  domain  as  the  project),  should  have  experience  in  the 
programming  language,  development  method  and  development  environment  to  be  used,  should 
have  been  trained  in  using  2167A  and  should  have  u.scd  the  .standard  in  a  previous  development 
task.  This  happy  state  is  rarely  achievable.  The  technical  software  manager  and  the  senior 
designers  should  have  this  level  of  expertise,  however,  before  the  project  .starts.  Other  members 
of  the  development  team  must  have  appropriate  computing  qualifications,  and  must  be  given 
appropriate  training  in  the  language,  methods  and  environment,  and  in  the  use  of  relevant 
standards  including  2167A. 

The  customer’s  technical  team  for  a  large  .software  project  should  have  at  least  one  person  with 
the  level  indicated  above  for  the  developer’s  senior  designers.  It  may  be  adequate  in  small  to 
medium  software  projects  for  the  project  office  to  rely  on  external  expert  advi.sers,  but  at  least 
one  person  in  the  day  to  day  management  of  the  project  must  have  experience  and 
understanding  of  the  use  of  the  relevant  standards  and  software  engineering  management  and 
methods. 

In  some  cases,  the  devclofX'r  may  need  to  educate  the  customer,  particularly  in  the  development 
method  and  CASE  tools  used.  This  may  involve  additional  cost  to  the  project,  in  both  time  and 
money. 

I'he  authors  recognise  that  the  pool  of  experienced  software  engineers  in  Au.stralia  is  limited  by 
the  relatively  small  si/e  of  the  dcfencc/aerospacc  software  market  and  that  low  mobility 
exacerbates  the  problem.  Nevertheless,  developers  and  customers  must  place  a  high  priority  on 
the  recruitment  and  retention  of  appropriately  skilled  staff. 

.T2  (.'ommunication  and  c(H)perali()n 

fhere  arc  many  reasons  supporting  a  close  woriring  rclalioaship  between  developer  and 
customer  ISPR90|,  but  the  authors  arc  aware  of  few  projects  where  such  a  relationship  exists. 

In  some  cases  the  customer’s  team  does  not  have  the  experience  to  support  such  a  relationship 
and  the  developer  is  understandably  reluctant  to  provide  the  needed  education  1MEY891. 
Developers  arc  also  wary  of  the  visibility  that  informal  communication  might  bring,  and  the 
potential  for  .subsequent  critic  .sm  of  their  development  procc.ss. 

The  benefits  of  increased  communication  and  cooperation  between  developer  and  cusiomer  arc 
seen  to  be: 


a  Better  mutual  understanding  of  each  others’  problems  and  a  subsequent  increase  in 
mutual  trust.  The  benefits  of  this  wilt  also  be  shown  by  more  meaningful  and  m.^rc 
successful  negotiations  and  reviews  |FIS871. 

b.  Improved  risk  management.  Risk  areas  can  be  highlighted  earlier  and  may  be 
assessed  by  both  customer  and  developer  before  a  solution  is  proposed. 

c.  Fewer  misunderstarxiings  and  misinterpretations.  Problems  are  identified  earlier  and, 
when  they  cannot  be  resolved  informally,  may  be  flagged  for  prompt  formal 
resolution. 
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(1.  Decreased  development  elTort  because  or  the  reduced  likelihood  of  a  course  of  action 
bcinji  rejected  by  the  customer. 

c.  Higher  quality  documentation  (at  least  from  the  customer's  point  of  view). 

Discussions  should  establish  the  form  of  documentation  and  the  level  of  detail  that 
the  customer  requires. 

fhe  risk  of  poor  communication  will  be  higher  when  there  are  many  contractors  developing 
software,  and  where  the  prime  contractor  is  not  a  major  software  developer.  In  these  cases 
special  consideration  needs  to  be  given  to  enhancing  the  communication  between  the  customer 
and  the  actual  developer  of  the  software. 

Meyer  el  al.  (Mt;V89|  slate; 

The  consistent  trust  and  goodwill  of  the  customer  towards  your  design  effort  will  be 
the  single  most  important  factor  in  your  ability  to  complete  your  project  on  time  and 
within  budget. 

3.3  Development  methodology 

In  soliware  engineering,  the  icmi  ’methodology”  refers  to  a  combination  of  life-cycle  models 
(paradigms),  management  practices,  technical  practices,  tools  and  training  procedures  used  to 
produce  software  |DEG‘){)|.  There  arc  many  different  models,  practices,  tools  and  procedures  to 
cIkxisc  from,  so  there  arc  many  diflercnt  development  mcthtxiologics.  each  with  certain 
strengths  and  weaknesses. 

The  methodology  selected  for  a  particular  software  development  project  must  be  .suited  to  the 
application  domain,  the  implementation  language,  the  magniiudc  of  the  development  effort  and 
other  characteristics  of  the  project  and  the  development  organisation.  In  addition,  if  the 
development  is  to  be  conducted  in  accordance  with  DOD-STD-2167A  then  the  development 
methodology  must  map  sensibly  into  2167A.  With  appropriate  tailoring.  2167A  can  be  forced 
to  accommodate  almost  any  methodology,  but  it  is  pointless  to  force  fit  a  methodology  that 
docs  not  support  the  basic  principles  of  2167A  (orderly  development  process,  visibility,  reviews 
and  audits)  or  that  is  not  suitable  for  programming  in  the  large. 

The  development  methodology  must  be  selected  quite  early  in  the  project  (such  as  at  tender 
preparation  time,  or  equivalent  for  in-house  projects).  The  methodology  must  be  written  down 
and  it  must  be  approved  by  the  customer  before  development  begins  |MEY89|.  The 
appropriate  place  for  specifying  the  methodology  (usually  by  referencing  separate  documents)  is 
in  the  preliminary  Software  Development  Plan  (SDP).  Where  a  significant  amount  of  time 
elapses  between  tender  and  contract  award,  it  may  be  nece.ssary  to  redefine  liic  development 
methodology  during  the  contract  negotiation  activities. 

Clearly  the  developer's  staff  must  be  well-versed  in  the  methodology  and  there  should  be  a 
training  plan  to  ensure  that  they  arc.  It  is  also  very  important  that  the  customer's  technical  staff 
have  an  understanding  of  the  methodology,  and  this  may  well  require  training  for  them  also. 

This  issue  is  discus.scd  in  further  detail  in  Section  7. 
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4  PRINCIPLES  OF  TAILORING 


4.1  (Jeneral 

I  ho  main  reasons  tor  tailoring  arc  as  follows: 

•  To  reduce  the  development  cflort  required  and  hence  to  reduce  the  cost  and  time  to 
completion. 

•  To  modify  or  add  requirements  which  are  needed  for  a  particular  project  or  software 
development  method. 

•  To  improve  the  quality  of  tlic  product  and  the  documentation 

•  To  correct  inconsistencies  and  hence  to  reduce  risk  in  the  contractual  process. 

In  the  authors'  opinion,  tailoring  is  essential  for  all  2167A  projects. 

failoring  requires  a  good  understanding  of  the  purpose  and  details  of  DOD-STD-2 167A  and 
cxc>eriencc  in  software  engineering  practices  and  management.  Tailoring  for  a  specific  project 
requires  further  understanding  of  the  application,  the  project  rcquircmeiiLs  and  (for  detailed 
tailoring)  the  development  method  used.  A  comprehensive  understanding  of  other  standards 
(particularly  MIL-,STD-I.S21B)  is  also  required  if  these  standards  arc  to  be  used  in  the  project. 

failoring  is  difllcult,  time  consuming  and  requires  a  meticulous  approach.  It  is  essentially  an 
iterative  process  -  each  proposed  change  must  be  assessed  with  regard  to  other  changes  and  its 
elfcet  on  the  management  and  development  process  as  defined  in  DOD-STD-2167A  and  other 
relevant  standards. 

Uncontrolled,  simplistic  or  inexperienced  tailoring  is  likely  to  lead  to  potentially  serious 
problems  as  follows: 

•  Loss  of  needed  visibility  to  the  customer. 

•  Omission  of  critical  requirements. 

•  Inconsistencies  between  requirements,  including  tho.se  in  other  standards. 

•  Documentation  which  is  inadequate  or  of  poor  quality 

failoring  .should  be  refined  throughout  the  course  of  a  project  (248 A,  287 1,  The  Request  for 
fender  (RFT)  should  address  the  high  level  tailoring  of  all  relevant  standards  and  specifications 
based  on  the  standards  chosen,  the  application  and  the  acquisition  process  The  RFT  should 
also  state  the  customer's  tailoring  policy  and  solicit  recommendations  from  tenderers  with 
icgard  to  cost  effective  tailoring.  The  detailed  tailoring  should  be  agreed  during  contract 
negotiations  and  included  in  the  contract.  Further  tailoring  will  often  be  nccc,s.sary  during  the 
course  of  the  contract.  In  ca.scs  where  the  need  for  refinement  during  the  development  can  be 
predicted,  plans  for  modifying  the  tailoring  should  be  included  in  the  SDP. 

It  is  recommended  that,  where  possible,  tailoring  is  carried  out  as  a  joint  effort  between  the 
customer  and  developer  1287).  This  will  ensure  that  each  is  aware  of  the  justification  and 
coasequcnccs  of  each  individual  tailoring,  as  well  as  encouraging  an  atmosphere  of 
communication  and  cooperation  (sec  also  Section  .1.2). 
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4.2  Tailoring  references  and  tools 

General  principles  for  the  tailoring  of  military  standards  and  specifications  arc  addressed  in 
DOD-HDBK-248A,  which  discu.sscs  the  advantages,  dangers  and  mechanisms  of  tailoring, 
MIL-HDBK-287  provides  a  more  detailed  approach  to  tailoring  DOD-STD-2I67A,  and  provides 
good  advice  on  the  relationship  of  2167A  to  other  standards  and  the  handling  of  the  '  shell " 
requirements.  Both  of  these  handbooks  are  recommended  to  tho.se  responsible  for  tailoring  and 
in  particular  to  customers  responsible  for  the  approval  of  tailoring. 

Logicon’s  Tailor  computer  based  tools  lLOG9()l  are  also  useful,  especially  for  the 
documentation  of  the  tailoring  and  the  preparation  of  CDRLs.  They  have  been  found  to  be 
particularly  useful  in  collaborative  tailoring,  where  the  customer  and  developer  draft  the 
tailoring  together,  using  the  tools  as  a  basis.  These  tools  are  mainly  aimed  at  tailoring  for  the 
full  scale  development  phase  of  a  project  and  arc  less  useful  for  the  tailoring  of  investigatory  or 
research  projects, 

4.3  Project  factors  influencing  tailoring 

Each  project  is  different.  DOD-SrD-2I67A  provides  requirements  which  arc  applicable  for 
many  projects,  but  the  requirements  should  be  tailored  to  meet  specific  project  factors.  Factors 
which  may  influence  tailoring  include; 

•  The  general  application  area  for  the  .software,  eg  operational,  prototype,  feasibility 
demonstration,  analysis,  test  or  training. 

•  The  type  of  software  being  developed  -  management  information  systems  and  real¬ 
time  combat  systems,  for  example,  should  require  different  tailorings. 

•  The  development  method  and  CASE  tools  used. 

•  The  use  of  non  procedural  languages  or  application  specific  program  generation  tools 
(such  as  4GLs). 

•  The  difficulty  or  complexity  of  the  ta.sk. 

•  The  number  and  complexity  of  external  and  intcr-CSCI  interfaces. 

•  Whether  parts  of  the  software  arc  safely  critical  or  security  critical. 

•  The  proportion  of  software  which  is  reused  from  a  previous  project  and  the  amount 
of  non-dcvciopmcntal  and  COTS  (commercial  off  the  shclO  software. 

•  The  level  of  software  support  that  will  be  required  cither  by  the  customer  or 
developer. 

•  The  verification  and  validation  (V&V)  approach,  including  whether  this  function  will 
be  performed  internally  or  by  an  independent  or  customer  ba.scd  V&V  team. 

4.4  .Staying  consistent  with  other  standards 

DOD-STD-2167A,  as  it  .stands,  is  incon.sistcnt  with  some  U  S.  military  standards  ([287 
Appendix  Bl).  Tailoring  2167A  and/or  the  standards  in  conflict  is  required  to  resolve  these 
inconsistencies.  In  particular  MIL-.STD-1.S21B  was  written  to  correspond  to  DOD- STD-2167 


I 


II 


ERL-0637-RE 


and  has  numerous  references  which  arc  incorrect  when  this  standard  is  used  with 
D()D-STD-2167A. 

Each  change  to  2167A  or  the  Data  Item  Descriptions  (DIDs)  must  be  considered  not  only  with 
respect  to  the  internal  consistency  of  2 167 A,  but  also  with  respect  to  other  required  standards. 

fhe  problem  of  inconsistency  between  standards  will  always  be  a  contractual  risk  area 
particularly  when  standards  arc  released  at  different  limes.  Where  possible,  customers  should 
spccincatly  nominate  the  version  of  standards  which  arc  to  apply  to  reduce  misunderstandings 
in  this  area. 

An  understanding  of  the  value  and  purpose  of  each  standard  specified  is  important  in  preventing 
iticonsisicncies.  Customers  should  avoid  ific  "shotgun"  approach  of  specifying  as  many 
standards  as  possible  in  the  hope  that  this  will  provide  added  protection.  Not  only  is  this  likely 
to  guarantee  inconsistencies  between  the  standards,  but  may  al.so  lead  to  additional  co.sls. 

IV11E-STD-1521B  is  likely  to  present  the  most  problems  regarding  conllicts.  The  sequence  of 
aciix  ilics,  documentation  delivery  and  reviews  of  2167A  arc  lightly  interwoven  with  the 
requirements  of  152 IB.  Deferring  the  delivery  of  a  document  to  a  later  stage  of  the 
development,  for  example,  may  mean  that  it  is  not  available  for  review  as  required  by  1521 B. 
and  that  there  arc  no  requirements  to  review  it  at  later  reviews.  Deleting  the  requirements  for  a 
particular  review  may  result  in  some  activities  or  documents  not  being  reviewed  at  all. 

4.5  Add,  delete  or  modify? 

D(fD-STD-2167A  (para.  1..5)  states  that  "the  tailoring  pitrcess  intended  for  this  standard  is  the 
deletion  of  non-applicabic  requirements".  MIL-HDBK-287  (para.  4. 3. 1. a)  states  that  "For  DIDs. 
requirements  may  be  deleted  or  partially  deleted,  but  not  modified".  Neither  of  these 
restrictions  is  binding  on  Australian  users  of  2 167 A  and  it  is  evident  that  in  many  cases  an 
adequate  tailoring  will  not  be  possible  without  the  modification  of  requirements,  including  those 
(or  the  DIDs  1MAR88,  MCG9()|. 

The  authors  thcmforc  recommend  that  the  tailoring  of  2167A  and  its  DIDs  should  he 
accomplished  by  die  deletion,  addition  and  modification  of  requirements.  For  consistency 
between  Defence  projects  however,  it  is  important  that  the  .structure  and  paragraph  numbering 
of  Ixnh  the  tailored  requirements  and  the  required  documentation  be  preserved  as  far  as 
possible. 

4.6  Over-detailed  tailoring 

It  is  not  necessary  to  tailor  out  paragraphs  which  arc  obviously  inapplicable  to  individual 
software  elements  (eg  particular  CSUs)  but  which  may  be  applicable  to  others.  Multiple  or 
over-detailed  tailorings  are  difficult  for  documenters  to  follow,  difficult  for  those  refining  the 
tailoring  to  maintain,  and  arc  prone  to  error.  It  is  preferable  that  the  tailoring  concentrates  on 
the  major  variations  from  the  requirements  of  2167A  rather  than  the  recording  of  petty  changes 
which  will  have  no  effect  on  the  quality  of  the  software  and  its  documentation,  or  the  effort 
required  in  producing  the.se. 

4.7  The  cost  of  tailoring 

The  authors  believe  that  there  is  no  cheap  solution  to  the  tailoring  problem.  Projects  have 
widely  differing  requirements  with  regard  to  both  the  development  process  used  and  the 
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documentation  deliverables.  These  differences  must  be  accommodated  by  tailoring  to  reduce 
the  cost,  schedule  and  risk  and  to  improve  the  quality  of  tlie  product,  including  documentation. 

The  tailoring  of  2167A  and  its  DIDs  requires  high  quality  .staff  with  speciali.sed  knowledge  and 
experience,  staff  whose  services  arc  normally  in  strong  demand.  The  task  of  tailoring  is 
normally  time-consuming  and  may  require  weeks  rather  than  days  over  the  life  of  the  project. 

In  the  authors'  opinion,  however,  the  benefits  of  an  optimal  tailoring  are  likely  to  far  outweigh 
the  costs  for  all  but  the  smallest  projects. 


5  THE  REQUIREMENTS  OF  2167A 

This  section  discusses  the  use  and  tailoring  of  the  requirements  of  2167A,  excluding  those  for 
documentation  (DIDs). 

5.1  Is  it  all  needed? 

The  full  requirements  o(  2lb7A  can  he  very  costly  to  implement,  not  only  in  the  development 
effort  hut  also  through  the  monitoring  task  of  the  customer.  Many  activities  such  as  product 
evaluations  require  additional  effort  both  in  their  execution  and  in  their  recording  and  auditing. 

In  addition  to  the  general  guidelines  for  tailoring  (287,  24SA).  the  authors  suggest  that  where 
the  developer  considers  that  a  requirement  is  not  cost  effective  and  where  the  customer  (or  his 
representative)  does  not  have  the  means  or  ability  to  verify  that  the  requirement  is  met. 
consideration  should  be  given  to  the  deletion  of  that  requirement.  In  the  authors’  experience, 
developers  arc  (understandably)  reluctant  to  meet  requirements  that  they  consider  not  to  be  cost 
effective  and  the  subsequent  implementation  is  often  inadequate.  If  the  customer  is  not 
prepared  or  able  to  monitor  and  enforce  the  fulfilment  of  the  requirement,  there  is  no  advantage 
to  the  customer  or  developer  in  specifying  it.  and  there  is  little  point  in  the  customer  paying  llic 
additional  costs  involved. 

5.2  Reviews  and  audits 

Reviews  and  audits  as  specilied  in  MIL-STD-1521B  arc  the  milestones  and  often  the  crisis 
points  of  21f)7A  projects.  The  authors'  survey  and  workshops  |GAB91 1  did  not  cover  the  issue 
of  reviews  and  audits  in  any  detail  (an  oversight),  hut  revealed  concern  regarding  the  overlap 
and  inconsistencies  between  21b7A  and  I521B.  Many  of  the  problems  that  both  customers  and 
developers  had  experienced  with  2 1 67 A  culminated  in  difficulties  with  cu.stomcr  approval  at 
reviews,  typically  Preliminary  and  Critical  Design  Reviews  (PDRs  and  CDRs). 

It  is  evident  tliat  in  many  projects  reviews  arc  the  main  occasions  when  technical  discussion 
about  software  takes  place  between  the  customer  and  developer.  This  factor  is  likely  to  be 
responsible,  at  least  in  pan,  for  what  appears  to  be  a  relatively  low  rate  of  outright  approval  on 
first  presentation  in  design  reviews.  The  developer  prepares  for  such  a  review  with  little 
knowledge  of  what  the  customer  is  likely  to  approve.  The  customer,  on  the  other  hand,  is  often 
presented  with  information  for  the  first  time  (formally  or  informally).  The  outcome  is  almost 
inevitable: 


•  The  customer's  technical  representatives  are  overwhelmed  by  the  new  information 
!  and  time  is  wasted  during  the  actual  review  meeting  in  explanation  and  education. 

The  fact  that  the  developer  has  in  many  cases  provided  more  information  (often 
irrelevant  to  the  review)  than  is  needed  aggravates  rather  than  assists  in  this  situation. 
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Because  the  design  is  new  and  is  being  seen  lor  the  first  time,  faults  in  the  design  are 
found. 

Different  perceptions  of  project  requirements  and  the  use  of  21(i7A  result  in 
protracted  discussions  with  regard  to  interpretation  of  requirements. 

Important  areas  are  completely  ignored  due  to  lack  of  time. 

The  review  results  in  disapproval  or  reluctant  contingent  approval  (in  the  words  of 
152  IB). 


Such  reviews  cannot  be  regarded  as  productive.  Apart  from  the  fact  that  much  of  the 
discussion  is  at  a  lower  level  than  is  warranted  by  the  number  and  status  of  attendees,  the 
<ieveloper  has  wasted  effort  in  preparing  for  the  review  and  the  schedule  is  likely  to  be  affected 
as  a  result.  Momover,  many  of  the  real  issues  have  not  been  tackled,  possibly  reducing  the 
overall  quality  of  the  product.  A  lasting  side  effect  is  likely  to  be  an  increased  distrust  between 
the  two  parties. 

1  he  lollowing  recommendations  are  made  with  regard  to  reviews  and  audits. 

a.  The  number  of  surprises  at  these  events  should  be  minimised.  This  can  only  be 
achieved  through  greater  infonnal  communication  between  the  customer’s  and 
developer’s  technical  staff  (Section  3.2). 

b.  Customers  should  ensure  that  their  technical  team  is  adequate  in  number,  education 
and  espcrietice  to  liaise  effectively  with  the  developer’s  team  and  review  delivered 
documentation 

c.  The  reviews  should  concentrate  on  higher  level  issues  -  interpretation  of  requirements 
should  have  been  resolved  prior  to  the  review. 

d.  For  largo  projects,  the  use  of  incremental  reviews  should  be  considered,  reviewing 
components  of  the  product  as  they  arc  produced,  and  summarising  the  resuhs  of  these 
at  a  (later)  formal  review  meeting. 

e.  The  reviews  and  audits  should  be  planned  and  Ihcir  requirements  tailored  in 
accordance  with  the  particular  characteristics  of  the  project,  including  the  application, 
the  project  si/c  and  complexity,  and  the  development  method  used.  The  planning 
should  include  the  number,  timing  and  nature  of  reviews. 

f.  Reviews  should  avoid  protracted  presentations  ("dog  and  pony  shows  ’)  of 
information  that  is  fully  documented  and  concentrate  on  issues  such  as  the  inherent 
design  of  the  software,  work  done,  progress  made  and  areas  of  potential  risk. 

g.  The  demonstration  of  user  interface  mockups  and  other  prototypes,  and  the 
capabilities  of  incremental  builds  .should  feature  prominently  in  the  appropriate 
reviews. 

In  many  smaller  projects  in  Australia  the  complete  .set  of  reviews  cannot  be  justified  and  is  not 

required  by  the  contract.  Tailoring  152 IB  and  2167A  to  remove  the  requirements  for  these 

reviews  is  relatively  straightforward  but  gives  rise  to  a  common  tailoring  problem:  deleting  a  I 

review  may  result  in  the  removal  of  review  requirements  that  arc  needed.  For  example, 

removal  of  the  Software  Specification  Review  (SSR)  removes  the  requirement  to  review 
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I  qualificalion  and  quality  factor  requirements  of  CSCIs.  which  arc  not  subsequently  reviewed  in 

the  PDR.  When  reviews  arc  tailored  out  their  requirements  should  be  analysed  and  those  that 
arc  required  should  be  added  to  a  subsequent  review 

5.3  Testing 

DOD-STD-2t67A  requires  testing  above  the  CSU  level  to  be  conducted  by  a  team  independent 
of  the  development  team.  This  requirement  entails  additional  cost  and  is  often  not  met, 
particularly  in  small  software  development  organi.sations.  In  the  authors'  opinion  this 
requirement  is  important  to  the  quality  of  testing  and  its  removal  can  only  be  justified  in 
exceptional  circumstances.  It  is  therefore  important  that  customers  (and  their  V«&V  agents)  pay 
particular  attention  to  ensuring  that  the  independence  of  testing  is  planned  and  implemented. 

The  authors  have  noticed  a  tendency  for  some  developers  to  attempt  to  postpone  the  definition 
of  formal  qualification  testing  to  a  later  stage  than  that  required  by  2167A  (in  some  eases  well 
after  CDR).  Presumably  this  is  motivated  both  by  a  desire  to  decrease  the  number  of  tasks 
during  preliminary  and  detailed  design,  and  to  defer  test  definition  until  the  performance  of  the 
system  is  known.  Whatever  the  reason,  it  is  neither  in  the  customer’s  or  developer’s  interests  to 
defer  test  definition. 

Springman  |SPR90|  indicates  one  developer’s  view  towards  test  definition  with 
recommendations  including: 

Defining  the  standards  and  procedures  for  testing  very  early  in  the  contract. 

•  Involving  the  customer  in  developing  the  test  program. 

•  Getting  the  customer  to  commit  to  a  cost  effective  test  program  at  an  early  stage. 

The  benefits  to  the  developer  arc  obvious  -  the  definition  (and  limiting)  of  testing  at  an  early 
stage  means  that  the  objectives  of  the  development  become  clearer  and  more  bounded.  From 
the  customer’s  point  of  view  the  test  details  indicate  that  the  developer  understands  the 
requirement  and  they  also  should  provide  a  clear  indication  of  the  projected  performance. 

If  the  developer  cannot  define  adequate  formal  qualification  tests  prior  to  the  CDR,  it  may  be 
an  indication  of  an  unstable  or  insufficiently  detailed  design. 

The  authors  strongly  recommend  the  definition  of  formal  testing  at  an  early  stage  in  the  contract 
and  no  later  than  that  specified  in  2 1 67 A. 

5.4  .Software  product  evaluations 

There  is  an  evident  lack  of  understanding  of  the  requirements  for  .software  product  evaluations 
in  current  2I67A  projects  (GAB91 1,  particularly  tho.se  where  thc.se  requirements  have  not  been 
tailored  out.  This  indicates  that  in  many  cases  product  evaluations  are  not  being  performed  or 
*  audited.  Product  evaluations  arc  conducted  by  the  developer  on  deliverable  software  and 

documentation,  prior  to  their  delivery  to  the  cu.stomer,  to  increase  the  quality  of  the 
deliverables.  The  evaluations  must  be  performed  by  a  team  independent  of  the  development 
team  and  records  must  be  maintained  of  the  evaluation  and  the  subsequent  corrective  actions. 

I  There  is  a  temptation  for  customers  to  requite  product  evaluations  as  a  form  of  compensation 

for  the  lack  of  the  appropriate  skills  in  their  project  teams.  The  authors  recommend  against  this 
for  two  reasons.  Firstly,  the  current  approach  to  product  evaluations  by  some  developers  would 
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rcsul(  in  no  langihic  increase  in  quality.  Scco'idly.  a  cu.stomcr  not  possessing  the  appropriate 
skills  would  not  be  able  to  audit  the  product  evaluation  process  adequately 

Product  evaluations,  if  they  meet  the  requirements  of  2167A.  require  a  large  amount  of  cITort 
on  the  developer’s  part,  and  an  additional  auditing  effort  from  the  customer.  It  is  likely  that 
c\[x.-ncnccd  developers  will  perform  some  form  of  evaluation  in  any  ca.se  as  part  of  their  V&V 
or  QA  process  -  it  is  more  cost  effective  and  efficient  to  detect  faults  in  software  and 
documentation  prior  to  delivery  rather  than  have  the  deliverables  rejected  by  the  customer. 

It  is  therefore  recommended  that  product  evaluations  be  required  only  on  projects  (and 
producls)  where  there  are  obvious  benefits,  and  where  the  customer  is  willing  and  able  to  audit 
the  product  evaluation  process.  These  should  include  projects  with  safety  or  security  critical 
software,  and  larger  projects  where  the  customer’s  technical  staff  cannot  monitor  all  of  the 
develo|)mcnt  in  detail. 

5.5  ('nnfiguration  management 

l)()D  SrD-21h7A  provides  its  own  requirements  (or  configuration  management  (although 
several  DlDs  refer  indirectly  to  MIL-STD  483).  Customers  should  ensure  that  Ihc  configuration 
management  requirements,  including  those  for  post-development  maintenance,  are  compatible 
with  those  of  2I67A  (tailored  as  needed)  and  that  any  specific  requirements  are  also  addressed. 

Many  developers  arc  concerned  about  the  adequacy  of  their  configuration  management  systems 
and  prtK'edurcs,  particularly  with  regard  to  their  ability  to  control  more  than  one  version  t'f  the 
software  satisfactorily  at  the  same  time.  Customers  should  be  aware  of  this  probicnr, 
particularly  for  large  or  complex  systems,  and  be  prepared  to  ensure  that  the  develo|X’t's 
configuration  management  procedures  are  both  adequate  and  adhered  to. 


6  PARTITIONINC;  THE  SYSTEM  INTO  SOFTWARE  ELEMENTS 


Mistakes  made  in  the  partitioning  of  systems  into  CSCls,  and  Ihc  subsequent  partitioning  of 
CSCls  into  CSCs  and  CSUs,  have  been  the  cause  of  numerous  difficulties  in  2Ui7A  projects 
lltlil.SS,  MAR88,  MCG9().  MEtTSyi.  The  partitioning  problem  is  not  simply  one  of  providing 
the  most  elegant  or  manageable  modularisation  based  on  analysis  and  design  factors 
Developers  must  also  consider  Ihc  clicci  of  partitioning  on  how  they  will  meet  the  requirements 
of  2lfr7A.  particularly  with  regard  to  review,  test  and  documentation  IXTming  tr>o  many 
CSCls  will  result  in  too  many  reviews,  formal  tests  and  documents;  if  the  software  is 
insufficiently  partitioned  it 's  likely  to  be  difficult  to  manage  and  provide  insufficient  visibility 
for  the  customer. 


I  here  can  be  no  specific  nilcs  for  the  selection  of  CSCls,  CSCs  and  CSUs  -  each  project  must 
make  its  own  judgements  in  this  matter.  The  following  suggestions  may  be  helplul 

6.1  Selecting  CSCls 

A  CSCl  is  first  and  foremost  a  configuration  item.  MIL-STD  483A  provides  sound  guidance 
for  the  selection  of  CSCls  and  states: 

The  selection  of  hardware/software  to  be  managed  as  configuration  items  should  be 
determined  by  the  need  to  control  a  configuration  item  s  inherent  characteristics  or  to 
control  that  configuration  item  s  interface  with  other  configuration  items.  The  selection 
is  a  management  decision  normally  accomplished  through  the  system  engineering 
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process  in  conjunction  with  configuration  management  and  with  the  participation  of 
logistics.  Selecting  configuration  items  should  be  with  a  full  view  of  the  hie  cycle  cost 
and  management  impacts  associated  with  such  a  designation  Choosing  too  many 
configuration  items  increases  the  cost  of  control:  choosing  too  few  or  the  wrong 
elements  as  configuration  items  runs  the  risk  ot  too  little  control  through  lack  of 
management  visibility. 

In  other  words  CSCls  need  to  be  selected  on  the  basis  of  the  capability  to  manage  and  control 
them.  Consideration  needs  to  be  given  to  the  following  leatures: 

•  Si/e  and  complexity  -  whether  the  CSCl  is  too  large  or  the  development  too  complex 
to  be  effectively  managed. 

•  Interfaces  CSCls  should  l)c  chosen  to  minimise  the  interfaces  to  other  CSCls  and 
MWCls  (Hardware  Configuration  Items).  This  is  particularly  stres.sed  by  Bulcy  ct  al 
[niii.«xi 

•  Fhinelionalit\  functional  boundaries,  including  operational,  training  and  support. 

•  Cniicalily  sfK.'ci.,l  management  may  be  retjuired  tor  selected  elements  on  the  basis 
of  sc'cuiiis.  saleiN  and  other  critical  factors 

•  Cc'nii.iciual  or  gec'grapliic  issues  -  it  may  be  unwise  for  a  single  CSCl  to  contain 
software  developed  by  moie  than  one  contractor,  at  separate  locations,  or  at  dilTeretii 
stages  of  a  proieci, 

•  Post  develo(imcni  use  and  maintenance  -  whether  software  is  to  be  used  or 
maintained  by  separate  agencies  alter  devch'pment 

•  Software  environment  whether  different  elements  of  the  software  will  run  on 
dillcrcnt  processors,  for  example 

•  dfte  likelihood  ('I  reuse  if  software  is  likely  to  be  reused  in  subsequent  projects  it 
may  be  useful  to  dcline  it  in  separate  ('SCIs  or  CSCs. 

Some  partitions  suggest  tfiemselves  A  program  running  on  a  single  processor  with  rcasonahiv 
common  functions  is  a  natural  candidate  for  a  CSCl  If  it  is  very  large  or  there  are  major 
divisions  in  its  functionality,  consideration  needs  to  given  to  splitting  it  into  two  or  moa' 

C’SCIs.  This  dtx’s  not  mean,  fiowevcr.  that  a  CSCl  sltould  only  contain  one  program 
Collection  of  a  group  of  smaller  programs  with  a  common  function  into  a  CSCl.  test  and 
diagnostic  programs  for  example,  is  sensible  practice.  It  also  docs  not  preclude  sollware  in 
diflcmtit  processors,  or  different  processor  types,  or  software  written  in  different  languages,  or  a 
mixture  of  new  and  existing  software  from  being  aggregated  in  a  single  CSCl  under  the 
appropriate  circumstances.  In  this  latter  case  the  tailoring  for  different  CSCs  in  the  SDl)  will 
need  to  be  considered  (see  Section  S  3  2) 

Both  developers  and  customers  should  be  aiming  to  minimise  the  number  of  CSCls  within  the 
constraints  of  manageability  and  visibility. 

6.2  .Selecting  CSCs  and  CSCs 

Selecting  the  CSCs  and  CSUs  is  also  a  critical  activity.  If  the  level  at  which  CSUs  arc  defined 
is  too  low.  the  si/c  and  content  of  the  CSCI's  Software  Design  Dwiiment  (SDD)  can  be  too 


17 


ERL-0637-RE 


h>  an  order  ol  magnitude.  It  the  level  is  too  high,  the  customer  will  be  denied  the 
'.iMbilil)  needed  to  review  and  approve  the  design. 

\  CSC  should  be  regarded  as  a  logical  lather  than  a  physical  entity  -  its  physical  realisation  is 
ilie  sei  ol  (’Si's  which  provide  its  capability  (and  which  may  be  grouped  in  sub-level  CSC’s). 
\Mtde  the  selection  ol  CSCs  will  olten  be  a  nalura'  outcome  of  the  design  method  used, 
consideration  of  the  documctilation,  integration  and  testing  requirements  of  the  (,'SC’s  is 
imj-Kiriant  DOD-.STD  2 167A  requires  that  each  CSC  be  described  in  terms  of  its  execution 
ct'nirt'l  atid  data  How  (among  other  things)  and  that  integration  and  testing  within  a  CSCI  be 
t'eilimned  on  the  basis  of  CSCs.  Incorrect  selection  of  CSCs  may  make  these  activities  quite 
diHiculi  and  time  consuming,  as  well  as  leading  to  poor  documentation. 

I  he  onh  guidance  that  2lb7A  provides  with  respect  to  the  selection  of  C.SLIs  is  the  dclinition 
I'!  a  CSC  as  being  "separately  testable”;  additional  clues  might  also  be  inferred  from  the 
inloniiaiion  required  in  the  SDO  riiis  has  caused  some  confusion  among  developers  ;ind 
ciiNii'mers  because  ci|  diHerent  interpretations  Many  have  interpreted  this  to  mean  (at  least 
iiuhalK  I  that  a  CSC  is  a  subirrogram  or  its  ecjuivalent,  giving  rise  to  an  enomious  amount  ol 
olien  unnecessary  documentation  This  intcrpa’tation  is  in  conllict  however  with  the 
lequiremenis  for  conliguralion  management  ;ifier  coding  and  unit  testing  |2lh7A  .‘S  .S  .'i); 

I  he  contrac'or  shall  place  the  source  cooe  for  each  successfully  tested  and  evaluated 
eSU  under  configuration  control. 

miplying  that  a  CSC  enctunpasses  one  or  more  source  code  entities  which,  at  least  in  modem 
'ohware  ileveU'pmetit.  will  olten  compnse  more  than  one  subprogram.  There  is  also  a  valid 
ai'Mimeni  that  a  single  program  in  a  source  Hie  containing  other  subprograms  cannot  be 
genuinely  "separately  testable  " 

I  his  IS  certainly  ati  area  ol  confusion  but  it  is  not  one  which  will  be  satisfactorily  resolved  by 
searching  standards  for  references  as  above.  The  important  issue  is  that  the  design  is 
documented  to  a  satisfactory  level  and  that  bmh  developer  and  customer  (and  the  mainlainer  in 
s.'ine  c.isesi  agree  as  to  the  meaning  td  '  salisfactoty'". 


I  he  authors  K'heve  that  a  CSC  shouhl  map  to  one  or  more  source  (lies  containing  one  or  more 
subprograms  or  similar  entities  which  are  closely  related  examples  that  meet  this  criterion  ate: 

•  .An  .Ada  package  (nomially  two  or  more  source  files). 

•  .‘N  small  library  of  loosely  related  lunciions  (such  as  a  malhematical  package). 

•  A  group  of  data  or  type  dcliniiions 

•  A  database  of  limited  complexity. 

•  One  or  more  object  class  delinitions 

(  sing  this  approach  it  is  still  likely  that  disagreements  will  occur  as  to  the  level  ol  complexity 
illowed  within  a  CSC.  Ihc  developer  s  written  software  development  method  should  address 
this  issue,  with  additional  written  agreements  with  the  customer  as  necessary' 

Meyer  et  al.  |MF.YS91.  Villc  ct  al  |V|L9()|  and  McCann  jMCCWOI  provide  case  .studies  of  CSC 
,md  CSC  selection  for  Ada  projects 
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7  DKVELOPMENT  ISSUES 

Ouriiig  the  authors’  surv'cy  ol  2167A  usage,  both  customers  and  developers  highlighted  a 
number  o(  issues  relating  to  the  relationship  between  software  development  practices  and 
2I67A  The  issues  can  be  categorised  into  two  main  areas:  those  associated  with  the  overall 
developmental  prtKcss  and  tliosc  associated  with  methods  used  during  the  process  such  as 
speciiic  analysis  and  design  methods 

7  1  The  Development  Process 

A  development  process  is  the  overall  framework  in  which  software  is  developed.  This 
Iramework  encompasses  the  various  activities  required  during  software  development,  such  as 
analysis,  desigti  and  testing.  Several  models  have  been  proposed  to  help  define  this  framework, 
the  ssaierlall  model  being  [xuhaps  titc  most  widely  discussed  (and  critici.scd)  of  the.se. 

l  ermiimldgy  tends  to  be  ,i  ma|or  stumbling  block  when  discussing  issues  relating  to  software 
process  m(«lel>  Most  discussions  focus  on  the  high-level  or  architectural  models  where  people 
t.ilk  in  terms  ol  waierlall,  spiral  IROFt'Hi,  incremental,  evolutionary  and  even  whirlpool  models 
inf'(i‘)t)l  T  he  situaiuMi  tx'C(imcs  cveti  more  confusing  if  the  many  variations  of  the  basic 
tiKHlels  are  consulcred.  for  esampic,  there  arc  several  variations  of  the  waterfall  model  rtinging 
(rom  a  strii  i  inierpreiaiion  (uherc  ifte  nc.xt  pha.se  cannot  start  until  the  previous  pha.se  is 
complete  I,  to  approaehes  where  software  is  developed  in  a  number  of  build  increments  (often 
tenned  mcretnetital  developmetU).  Statements  such  as  "2167A  is  incompatible  with 
evoluttoiiary  development '  Ixvome  mcaninglc,s.s  becau.se  individuals,  organi.sations  and  authors 
h;ive  dilTerent  prect'ticeived  ideas  as  to  what  the  term  means.  An  incremental  approach  might 
rightly  be  termed  evolutiotiary,  yet  others  sec  evolutionary  development  as  an  approach  where  a 
prototyix.'  is  evolved  into  a  final  produce  Confusion  abounds  when  this  terminology  is  also 
used  to  desciitx'  actjuisiiioti  processes.  For  example,  how  docs  evolutionary  acquisition  differ 
from  evolutiott.iry  development'.’  (’learly.  the  software  world  has  once  again  fallen  prey  to 
overuse  attd  ntistisc  of  lemiiiutlogy.  Piior  to  any  discussion  of  issues  relating  to  software 
processes  and  2I67A,  the  tcrtninology  must  be  clearly  defined.  Discussions  should  proceed 
based  on  a  consistent  understanding  of  specific  models,  rather  than  ill-defined  software  jargon. 

Applying  a  simple  label  such  as  "we  tire  using  incremental  development"  is  insufficient  to 
describe  the  overall  process  ol  software  development.  Published  models  may  provide  some 
general  idea  as  to  whtil  is  intended;  however,  they  rarely  lit  well  with  a  "real  world " 
development.  Programmers  often  remark  that  "the  watcriall  approach  has  been  specified  for  our 
project,  but  it’s  not  really  tlic  way  we  go  about  developing  .software".  If  this  is  the  case,  then 
what  is  the  actual  process  or  approttch  being  used'.'  How  docs  it  differ  from  the  standard 
waterfall  model  ’  Has  it  been  documented  .’  How  is  the  process  being  managed'.'  Indeed,  each 
project  has  specific  characteristics  and  it  would  be  inconceivable  that  a  standard  model  could 
define  the  process  for  all  projects.  For  example,  an  important  characteristic  that  needs  to  be 
considered  is  risk.  If  a  project  rctiuires  (lie  development  of  a  complex  user  interface,  then 
defining  the  requirements  for  the  interface  could  be  considered  to  be  a  high  risk  activity.  In 
this  ease,  user  interface  prototyping  might  be  considered  as  an  inclusion  into  the  development 
process  to  assist  in  controlling  the  risk.  Software  development  is  a  very  complex  endeavour.  It 
is  characterised  by  many  features  including  tool  usage,  type  of  project,  available  resources,  and 
(Ktssiblc  risks  Reference  to  a  simple  high-level  model  is  insufficient  to  describe  the  many 
complex  interactions  and  dependencies  that  ultimately  define  the  development  process, 

I'hc  2167A  standard  is  olten  cnticiscd  as  being  based  on  a  waterfall  approach.  Although  the 
waterfall  approach  is  used  as  a  model  to  help  convey  the  requirements  of  the  standard,  it  would 
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ho  fnolish  lo  believe  that  software  development  could  proceed  based  solely  on  the  model 
described  in  2167A.  In  fact,  the  standard  specillcally  slates  in  para  4.1.1  that 

The  contractor  shall  implement  a  process  tor  managing  the  development  of  the 
deliverable  software.  The  contractor's  software  development  process  shall  be 
compatible  with  the  contract  schedule  for  formal  reviews  and  audits. 

It  then  goes  on  lo  list  the  major  activities  which  can  be  applied  "iteratively  or  recursively." 
Notice  also  that  the  standard  slates  that  the  process  is  to  be  compatible  with  "the  contract 
schedule".  It  docs  not  specify  when  reviews,  audits  or  deliverables  arc  to  be  for  even  have  to 
be)  provided.  Indeed,  these  milestones  need  lo  be  defined  through  close  cooperation  between 
the  customer  and  developer.  They  will  invariably  be  heavily  dependent  upon  the  proce.ss 
defined  by  the  developer  and  agreed  tailorings. 

!t  is  possible  that  the  use  of  MIL  STD- 152 IB  is  more  responsible  than  2 167 A  for  forcing  a 
project  into  a  waterfall  approach.  There  arc  also  indications  that  .some  less  disciplined  software 
dcvclo[X'rs  regard  almost  any  controlled  process  as  a  waterfall  approach  because  of  the 
necessity  for  restrictions  imposed  by  the  control  mechanisms  (eg  the  need  to  undertake  design 
activities  before  coding,  and  the  need  to  document  the  design). 

.'\  major  failing  of  many  projects  is  that  the  development  process  is  at  best  ptxtrly  defined.  It  is 
loo  easy  to  blame  the  "waterfall  approach  imposed  by  the  standard"  as  the  reason  for  failure;  in 
most  ca.scs  it  is  really  due  to  a  lack  of  planning  and  a  lack  of  cooperation  between  the  customer 
and  developer.  Without  a  clear  direction  as  lo  how  development  will  proceed,  the  project  will 
become  plagued  with  cuslomcr/dcvclopcr  disputes  and  unexpected  surprises  generally  arising  at 
the  most  crucial  stages  of  development.  Customers  should  ensure  that  the  process  to  be  used  is 
well  defined,  understood,  and  documented.  Interaction  and  dependencies  between  the  various 
development  activities  should  he  provided  and  the  model  (as  specified  in  the  SDP)  should  be 
thoroughly  reviewed  and  analysed. 

I'he  development  process  must  be  built  around  the  fundamental  requirements  of  2167A.  with 
particular  regard  lo  those  requirements  providing  visibility  to  llic  customer  and  progress 
reporting.  To  propose  a  model  without  a  thorough  understanding  of  the  requirements  and 
implications  of  2167A  will  almost  certainly  result  in  conflicts.  There  have  been  many 
arguments  indicating  that  2167A  docs  not  fit  with  the  way  Uic  developer  likes  to  develop 
soliware  (generally  based  on  some  high  level  model).  The  authors  contend  that  a  process 
model  needs  to  be  elaborated  for  each  project  so  dial  specific  project  characteristics  can  be 
accommodated.  One  of  these  is  the  tailored  set  of  2I67A  requirements.  Humphrey  |HUM891 
proposes  an  approach  based  on  process  cells  which  would  support  elaboration  of  comprehensive 
project  related  process  models.  Indeed,  his  discussion  of  various  aspects  of  process  modelling 
is  recommended  reading.  Regardless  of  the  approach  taken,  tho.se  involved  in  developing  a 
process  model  for  a  2 167 A  project  must  have  a  good  understanding  of  the  various  approaches 
to  software  development,  a  solid  background  in  process  modelling,  a  knowledge  of  the  project 
and  product  characteristics,  and  an  expert  knowledge  of  2167A. 

7.2  Development  Methods 

rcciuircmcnt  of  2167A  (para  4.2.1)  .states  that  "the  contractor  shall  use  .systematic  and  well 
documented  .software  development  methods  to  perform  requirements  analysis,  design,  coding, 
integration  and  testing  of  the  deliverable  software".  No  .specific  methods  are  defined,  yet  some 
mclhotls  have  been  found  to  be  more  easily  documented  using  the  DID  formal  than  others. 
Questions  that  arise  include; 
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•  Which  mclhdcis  should  be  used  for  a  certain  class  of  project'.’ 

•  What  constitutes  a  "well  documented  method"? 

•  What  level  of  tailoring  is  required  of  2I67A  DIDs  for  specific  methods? 

There  arc  numerous  published  development  methods  -  they  range  from  academic  approaches 
which  lack  rigour  or  scalability,  to  more  rigorous  and  well  documented  methods  developed  by 
large  organisations.  In  addition,  there  are  several  variations  on  a  basic  theme.  For  example,  the 
words  "object  oriented"  are  used  to  describe  a  vast  range  of  different  methods.  A 
characterisation  of  all  the  available  methods  (and  their  tailoring  implications)  is  well  beyond  the 
scope  of  this  report;  the  following  guidelines  may  be  of  some  assistance,  however. 

A  common  question  is  "which  methods  would  most  suit  our  project"?  This  begs  the  question 
"what  arc  you  building?  "  Prior  to  assessing  the  applicability  of  methods,  the  project  must  be 
characterised;  is  it  real-time,  hard  real-time,  data  oriented?;  how  large  and  complex  is  it?; 
what  arc  Ihe  major  perceived  risks  ’  Other  aspects  to  consider  include  process  characteristics 
(eg  "does  the  organisation  intend  using  automated  tool  support?"),  interrelationships  with  other 
methods  (eg  "will  Structured  Analysis  fit  well  with  an  object  oriented  design?").  In  addition  to 
understanding  project  and  process  characteristics,  the  soliwarc  engineers  rc.sponsiblc  for  defining 
and  selecting  the  methods  must  have  a  .sound  knowledge  of  available  techniques  and  their 
application.  In  some  eases,  none  of  the  available  methods  will  ideally  suit  the  project  and 
elements  of  more  than  one  method  may  need  to  be  applied. 

Regardless  of  the  approach  taken,  the  method  must  be  well  documented.  Documentation  should 
include: 


•  A  description  of  the  general  philosophy  and  approach. 

•  A  description  of  the  graphical  notations  used. 

•  The  basic  steps  to  be  taken. 

•  The  metliod  of  recursion. 

•  Essential  DID  tailoring. 

•  Examples  showing  the  products  resulting  from  the  application  of  the  method. 

Development  methods  must  be  defined  early  in  the  process.  The  methods  to  be  u.sed  need  to  be 
thoroughly  reviewed  and  assessed  prior  to  software  development.  Some  aspects  that  need  to  be 
considered  include  compatibility  with  other  methods,  compatibility  with  the  overall  development 
process,  understandability ,  completeness,  consistency  and  tlie  effect  on  DID  tailoring. 

Atialysis  and  design  technitiues  cannot  be  Icit  to  the  discretion  of  individual  programmers; 
analysis  and  design  activities  should  be  conducted  according  to  well  defined  and  documented 
methods.  These  methods  should  be  specified  in  the  SDP  at  an  early  stage  of  the  project,  as 
required  by  2Ur7A.  216S  and  AS 

Clear  separation  of  the  analysis  and  design  activities  is  critical  (but  is  often  difficult  to  achieve 
with  inexperienced  staff).  In  particular,  the  .Software  RequiremenLs  Specification  (SRS)  .should 
contain  requirements  only,  and  avoid  any  software  design  information  |SPR90.  MCGW).  (This 
error  occurs  far  too  frequently  and  causes  a  protracted  SSR,  unduly  volatile  SRSs  and  increased 
documentation  costs.) 

Tailoring  of  the  DIDs  may  be  required  to  support  specific  methods.  The  degree  of  tailoring 
will  depend  on  tlie  method  used  and  the  project.  For  example,  the  SRS  is  arranged  in  terms  of 
capabilities  and  .scpaniic  data  element  requirement.s.  Capability  descriptions  arc  functionally 
based  and  required  details  of  inputs  and  outputs.  If  a  Structured  Analysis  approach  is  used, 
little  tailoring  of  the  SRS  would  be  required  since  the  DID  contents  and  format  correspond  to 
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this  type  of  approach.  The  use  of  an  object  oriented  analysis  technique,  however,  may  require 
significant  tailoring.  Here  the  aim  is  to  document  the  analysis  in  terms  of  objects,  where  an 
object  may  encapsulate  both  data  and  operations.  This  approach  has  been  used  very 
successfi’Ily  for  a  range  of  projects  and  should  not  be  dismi.sscd  simply  because  it  docs  not  (it 
well  with  the  DID  structure.  If  significant  DID  tailoring  is  required,  the  customer  and 
developer  must  work  together  for  the  best  solution.  The  developer  must  have  a  well  defined 
method,  educate  the  cuslomcr  in  the  method  and  proposed  tailoring,  and  show  how  the 
approach  best  suits  the  particular  development.  On  the  other  hand,  the  customer  must  be 
receptive  to  new  thinking  and  realise  that  there  is  a  likelihood  that  the  DlDs  will  require 
tailoring  for  specific  methods.  The  most  important  thing  is  that  the  customer  and  contractor 
must  collaborate  in  reaching  an  agreed  tailoring. 


8  DOCUMENTATION  ISSUES 

8.1  (Jeneral  recommendations 

The  documentation  required  by  the  2I67A  DIP;,  is  designed  to  be  an  integral  part  of  2 167 A 
developments.  Not  only  docs  each  document  complement  the  others,  but  each  has  its  place  in 
the  development  process.  Ignorance  of  the  relationships  between  the  documents  and  each 
document’s  role  in  the  process  can  lead  to  serious  increases  in  documentation  costs  as  well  as 
other  problems  in  the  development  process.  Lack  of  understanding  and  agreement  between  the 
customer  and  developer  as  to  what  is  required  in  the  documentation  will  also  lead  to 
unnecessary  work  and  delays  (sec  also  Section  5.2). 

One  simple  example  of  the  need  to  consider  each  document's  contents  in  the  framework  of  the 
documentation  family  is  the  definition  of  the  software  test  environment  in  the  Software  Test 
Plan  (STP).  In  most  projects  the  test  environment  is  a  subset  of  the  software  engineering 
environment  described  in  the  SDP  and  is  best  explained  in  that  context.  It  thereforc  makes 
sense  in  lhc.se  ca.scs  to  describe  the  test  environment  in  the  SDP  at  the  level  of  detail  required 
by  the  STP  and  to  refer  to  it  in  that  document  to  prevent  duplication. 

DOD-STD-2167A  requires  that  a  systematic  development  method  be  used  and  the  DlDs  rcllcct 
this.  More  importantly,  the  DlDs  assume,  rightly  or  wrongly,  a  structured  analysis  and  design 
process  based  on  functional  decomposition.  If  the  actual  method  used  docs  not  follow  this 
broad  model  the  mapping  of  the  analysis  and  design  to  the  DlDs  will  be  at  best  difficult  and  at 
worst  impossible,  causing  problems  both  in  die  preparation  of  documents  and  in  their 
acceptance  by  the  customer.  In  particular,  if  the  analysis  step  is  not  performed  (usually  an 
indication  of  the  developer  not  following  a  systematic  method)  the  SRS  and  IRS  (Software  and 
Interface  Requirements  Specifications)  will  be  difficult  to  write  and  will  be  almost  valueless. 

General  guidelines  for  the  preparation  of  2167A  documentation  arc  as  follows; 

a.  Always  consider  each  document's  role  in  the  documentation  family  and  the 
development  process. 

b.  Consider  the  potential  audience  of  each  document. 

c  Consider  the  costs  involved  in  changing  baselined  documents  when  writing  them. 

d.  Provide  no  more  information  in  each  document  than  is  necessary.  If  additional 
information  could  be  useful,  include  it  as  "information  only"  in  an  appendix  or  as  a 
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separate  document.  Avoid  the  temptation  of  dumping  all  the  information  currently 
available  into  a  requirements  or  design  document. 

c.  Avoid  duplication  which  may  lead  to  inconsistencies  when  changes  arc  made.  In 
many  cases  this  can  be  achieved  by  cross-referencing. 

f.  Avoid  the  documentation  of  design  details  in  requirements  documents. 

g.  Ensure  that  the  development  method  and  CASE  tools  used  are  compatible  with  the 
requirements  of  2 167 A  and  the  development  of  2167A  documentation. 

K.2  Documentation  on  electronic  media 

The  DIDs  arc  not  prescriptive  about  the  form  in  which  the  documents  arc  to  be  delivered.  Each 
DID  states: 

7.2  The  Contract  Data  Requirements  List  should  specify  whether  this  document  is  to 

be  prepared  and  delivered  on  bound  8  1/2  by  1 1  inch  bond  paper  or  electronic  media. 

If  electronic  media  is  selected,  the  precise  format  must  be  specified. 

While  the  authors  arc  not  aware  of  projects  in  Australia  where  the  documentation  is  being 
delivered  only  in  the  form  of  electronic  media,  it  is  sometimes  provided  in  addition  to  the 
printed  form. 

There  arc  advantages  in  using  electronic  media  for  documentation  including; 

•  Changes  may  be  quickly  made  to  the  documcnlulioti. 

•  The  documentation  may  be  transmitted  by  electronic  means.  This  is  an  advantage 
not  only  in  the  initial  delivery  but  also  for  distribution  among  geographically  separate 
members  of  the  custome.  s  team. 

•  Different  versions  of  the  same  documents  may  be  compared  easily  to  highlight 
changes. 

•  Document  users  may  use  software  tools  to  scan  documents  for  key  words. 

•  Storage  of  documentation,  particularly  multiple  versions,  is  more  clTicicnt. 

•  Costs  and  response  time  in  the  generation  and  distribution  of  documentation, 
particularly  where  a  large  number  of  copies  might  be  normally  required,  may  be 
educed. 

•  customer  may  more  effectively  maintain  the  documentation  throughout  the  life 
of  the  system. 

There  are  also  disadvantages: 

•  The  configuration  control  of  documentation  needs  special  attention  if  its  most 
common  form  is  electronic. 

•  Electronic  documentation  may  result  in  additional  direct  costs  to  the  customer,  both 
in  the  procurement  of  compatible  documentation  equipment  and  in  the  training  of 
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staff  (although  this  may  be  reduced  or  eliminated  by  the  specification  of  a 
(focumcntalion  system  already  in  use  in  the  customer  organisation). 

fhe  authors  recommend  that  customers  require  the  delivery  of  documents  in  electronic  form 
even  it  printed  copies  arc  also  required.  Requirements  for  printed  copies  should  take  into 
consideration  the  availability  of  electronic  copies  and  be  reduced  accordingly. 

In  detennining  the  acceptability  or  otherwise  of  the  format  of  documentatioti  on  electronic 
media,  the  following  should  lx-  taken  inlo  consideration; 

•  What  facilities  will  be  required  to  use  the  documentation  and  to  make  printed  copies. 
Consideration  should  include  other  customer  agencies  which  may  need  to  use  the 
documentation. 

•  Whether  the  formttl  wilt  allow  the  customer  to  make  changes  to  the  documentation 
such  as  inserting  comments,  or  updating  documents  during  .sotfware  mtiintenancc. 

•  Whether  the  format  and  tools  support  compari.son  of  document  versions. 

•  Whether  the  loimat  supports  documentation  in  graphic  fomi. 

•  The  physical  medium,  if  any,  for  delivery'. 

I  (u  Defence  projects,  the  customer  should  also  ensure  that  documentation  on  electronic  media 
'.''nloims  with  the  CALS  requirements  (Computer-aided  Acquisition  and  Logistics  Support). 

8..1  f  ailoring  (he  f)M)s 

In  tailoring  the  requirements  for  documentation,  the  aim  should  be  to  produce  the  minimum 
consistent  set  of  documents  which  meets  the  needs  of  the  project. 

(General 

Ihe  deletion  of  individual  DlDs  from  the  documentation  requirements  fora  project  is  often 
desirable  but  consideration  needs  to  be  given  to  the  lull  consequences  of  its  deletion. 

II  the  customer  intends  to  rely  on  the  developer  ttt  provide  support  for  die  softwtire.  for 
example,  there  may  be  no  need  for  a  Computer  Resources  Integrated  Support  Document 
if'RISD),  Computer  System  Operator's  Manual  tCSOM)  or  Software  Programmer's  Manual 
iSPM)  to  be  produced.  Customers  should  be  aware,  however,  that  there  are  several  levels  of 
supimrt  which  may  be  required.  Modem  software  is  often  configurable  and  the  reconfiguration 
may  be  quite  complex.  By  deleting  the  a'quiremenls  for  the  CRISD  and  SPM  the  nomial 
vehicle  for  the  supply  of  this  information  is  removed  (see  Section  9.8). 

Dcletitui  of  a  high  level  DID,  such  as  the  SSDD,  will  also  require  Ifie  tailoring  of  several  other 
DIDs.  The  table  below  shows  the  relationship  between  the  various  DlDs  and  standards, 
inijicaiing  references  from  one  document  to  another.  Although  not  all  of  the  references  arc 
significant  (in  some  cases  they  are  mainly  for  itifomiation  purposes)  tlic  table  shows  that  care 
must  be  taken  in  deleting  DlDs 

One  misapprehension  that  the  authors  have  noticed  in  several  projects  is  the  view  that  2167A  is 
all  encompassing  and  that  no  additional  documentation  requirements  are  necessary.  While  (his 
may  be  true  in  some  eases,  many  projects  require  additional  documentation  which  should  be 
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identified  in  the  CDRL,  cither  as  the  tailoring  of  specific  DIDs  or  as  separate  documents  (see 
also  section  2.6).  Furthermore,  good  software  engineering  practice  always  requires  the 
generation  of  diagrams  or  other  information  not  explicitly  specified  in  any  DID.  Examples 
include  user  interface  layouts,  concurrency  diagrams  and  timing  analyses.  These  should  all  be 
provided  somewhere  in  the  delivered  documentation. 

8.3.2  Adaptive  tailoring 

In  many  projects  a  single  tailoring  for  all  SRSs  and  SDDs  (say)  will  not  be  adequate.  Different 
requirements  for  the  software  development  will  result  in  the  need  for  different  tailorings  in 
some  eases.  It  will  also  be  necessary  in  some  circum.stanccs  to  apply  different  tailorings  for 
different  CSCs  or  CSUs,  for  example  when  there  is  a  combination  of  new  and  existing  software 
in  a  CSCI. 

8.3.3  Using  alternate  formats  in  DIDs 

The  DIDs  arc  deliberately  prescriptive  with  regard  to  the  stmciurc  (including  paragraph 
numbering)  and  content  of  documentation.  This  approach  is  intended  to  guarantee  the 
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coinpiclencss  of  tlic  documents  and  to  assist  reviewers  in  linding  specific  information.  Any 
cofisideration  of  tailoring  or  alternative  fomiats  must  take  this  into  account. 

Wliile  the  DIDs  control  the  content  and  layout  of  documentation  they  encourage  the 
piesciitation  of  information  in  formats  appropriate  to  the  content,  and  the  use  of  cross- 
rclerencing  rather  than  duplication.  Statements  such  as  the  following  (SDD  10. 1. 0.2. 2 1  arc 
relatively  common: 

This  information  may  be  provided  by  automated  tools  or  other  techniques,  such  as  a 
program  design  language,  flowcharts,  or  other  design  representations. 

I  he  authors  also  recommend  tlic  use  of  appendices  or  separate  documents  where  this  will 
improve  the  presentation  and  u.sc  of  infonnation.  This  can  be  done  in  many  cases  without  the 
need  lor  tailoring  as  long  as  the  basic  structure  of  the  document  remains  intact  and  coherent, 
the  extracted  information  is  at  a  relatively  low  level  (eg  paragraph  4.X.Y.2  in  the  SDD  which 
addresses  the  design  of  an  individual  CSfl),  and  the  sub.scqucj)l  references  are  direct  and 
unambiguous.  Such  a  practice  is  particularly  useful  for  bulky  material,  supporting  itilbnnation 
and  the  documentation  of  information  which  is  orthogonal  to  the  basic  structure  of  the 
document 

file  more  general  extension  of  this  practice,  ie  the  use  of  DIDs  as  shell  documents  with  (often 
hroadi  references  to  documents  written  to  the  developer’s  internal  standards,  should  be  avoided. 

S,4  Software  development  files 

Develt'iK'rs  are  required  to  "document  and  implement  procedures  for  establishing  and 
maintaining  SDFs”  |2167A  4.2.9|.  The  authors  view  SDFs  (soltware  development  files)  as  very' 
important  for  the  documentation  of  additional  development  infomiation  including  justification  of 
design  choices  and  alternative  considerations,  CSU  test  procedures,  and  other  information, 
sometimes  of  an  infonnal  nature,  that  may  assist  in  software  maintenance.  In  addition,  if  the 
SDD  is  tailored  to  the  extent  .suggested  in  Section  9.5,  the  SDFs  arc  critical  for  the 
documentation  of  lower  level  CSUs  and  the  support  of  SDD  documentation  during  CDRs.  This 
.ipprt'ach  follows  that  of  Springman  |SPRK91, 

I  he  SDFs  should  be  placed  under  conliguralion  control  following  the  successful  testing  of  their 
U'.sociated  software  element  (normally  CSU  or  CSC)  and  be  delivered  to  the  customer  if  the 
iranxitioning  of  software  supjxnl  is  required. 

1)01)  ST  D-2I67A  .should  be  tailored  to  include  these  requirements.  7'hc  customer  .should  afso 
ensure  that  the  developer  has  adequate  procedures  for  the  use  and  maintenance  of  SDFs,  and 
that  the  CDRL  includes  their  delivery  It  may  be  necessary  in  some  circumstances  to  negotiate 
the  protection  of  intellectual  pro[X’rty  contained  in  delivered  SDFs. 


9  THK  DATA  ITKM  DESCRIPTIONS  (DIDS) 

9.1  .Software  Development  Plan 

I  he  SDP  describes  the  organisation,  management  and  planning  of  the  software  development  for 
a  project. 

rite  effort  required  in  preparation  of  this  very  important  document  can  normally  be  reduced  by 
referencing  the  developer's  own  software  development  standards  wherever  possible.  Where  this 
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occurs  those  standards  must  be  provided  to  the  customer  for  the  review  of  the  SDP,  and  must 
be  available  to  the  development  team  before  development  starts.  It  may  be  preferable  (for  both 
customer  and  developer)  to  de.scribe  the  high  level  requirements  being  met  in  the  SDP  and  to 
use  references  to  show  how  those  requirements  are  met. 

One  area  of  definition  in  the  SDP  which  will  vary  greatly  between  different  projects  is  Risk 
management  (SDP  section  3.3).  Consideration  should  always  be  given  to  tailoring  this  .section 
for  the  specific  application  and  project. 

9.2  System/Segment  Design  Document 

The  SSDD  describes  the  system  (or  segment),  the  allocation  of  CSCIs  and  HWCls  and  the 
processing  resources.  Its  definition  a.ssumes  that  its  requirements  arc  derived  from  a 
System/Segment  Specification  (SSS)  as  defined  under  M1L-STD-49()A  and  it  should  be  tailored 
if  the  system  specification  is  a  Prime  Item  Development  Specification  (PIDS),  Critical  Item 
Development  Specification  (CIDS)  or  other  specification. 

Where  there  is  only  one  CSCl  and  the  operational  requirements  arc  well  defined,  the  SSDD 
may  not  be  necessary  in  its  entirety.  In  this  ca.se  it  may  be  tailored  out,  and  any  of  the  SSDD 
requirements  that  are  considered  relevant  may  be  added  to  the  SRS. 

When  preparing  sy.stem  specifications,  it  is  important  that  customers  aim  to  avoid  restricting  the 
design  of  the  system.  Over-specification  of  requirements  can  make  the  analysis  and  design  task 
more  difficult,  as  well  as  forcing  design  details  into  requirements  specifications  such  as  the 
SRS. 


9.3  Interface  documents  (IRS  and  IDD) 

The  authors’  survey  revealed  a  level  of  dissatisfaction  with  duplication  between  the  IRS  and 
IDD  and  the  usefulness  of  both  documents.  In  the  authors’  experience  these  documents  arc 
often  misunderstood  and  subsequently  misu.scd.  The  wording  of  the  DlDs  does  not  assist  in 
this  regard. 

An  IRS  is  used  to  specify  the  requirements  for  one  or  more  CSCI  s  external  interfaces,  the 
details  of  which  arc  necessary  for  the  preliminary  and  detailed  design  of  the  system.  Interfaces 
may  be  to  other  CSCIs,  HWCls  or  external  to  the  sy.stem.  After  SSR  the  IRS  becomes  part  of 
the  allocated  baseline.  The  level  of  information  provided  in  the  IRS  should  mflcct  the 
interface’s  criticality  in  the  design  of  CSCIs.  Detailed  interface  information  should  only  be 
specified  for  interfaces  which  arc  external  to  the  system,  whose  definition  is  critical  to  the 
design  of  CSCIs,  or  which  arc  defined  and  arc  unlikely  to  change  (such  as  the  interface  to  an 
existing  HWCI).  Other  interfaces  should  be  identified  and  specified  in  terms  of  their 
requirements.  It  should  contain  no  design  information. 

To  some  extent  this  is  a  trade-off.  Information  that  is  likely  to  change  during  design  should  not 
be  in  an  IRS  -  as  it  is  part  of  the  allocated  ba.sclinc  changes  can  be  costly.  On  the  other  hand 
the  design  of  a  CSCl  requires  the  definition  of  external  interfaces,  and  instability  in  its  external 
interface  definitions  is  a  source  of  risk.  The  customer  and  developer  must  therefore  agree  on 
which  interfaces  must  be  defined  in  the  IRS  and  which  must  be  "designed "  and  expanded  in  the 
IDD, 

The  IRS  and  IDD  can  therefore  be  regarded  as  containing  the  appropriate  level  of  interface 
definitions  at  PDR  and  CDR  respectively.  The  IDD,  being  a  derivative  of  the  IRS.  must 
inevitably  duplicate  some  of  the  information  contained  therein,  but  this  problem  will  be  reduced 
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il  ihe  IRS  coniains  Ihc  correct  level  ol  information  and  there  arc  no  major  instabilities  in  the 
overall  system  design.  It  can  be  further  reduced  by  referencing  the  IRS  from  the  IDD,  where 
Ihe  IRS  contains  detailed  information,  although  il  may  be  preferable  to  include  all  the  interface 
information  in  one  document,  which  should  be  the  IDD. 

In  many  smaller  projects  a  single  interface  document  will  .suffice.  This  will  either  be  the  IRS 
or  IDD  depending  on  the  relevant  stability  of  interfaces  external  to  the  system  and  those 
iK’iwecn  rSCIs.  In  these  ca.scs  tailoring  must  be  used  to  delete  the  requirements  for  die 
discarded  document,  and  also  to  compensate  for  its  absence.  The  configuration  management 
requirements  must  also  be  tailored  accordingly. 

riieie  arc  also  situations  where  neither  an  IRS  nor  IDD  is  needed.  Where  a  project  consists  of 
;i  single  CSCl  running  as  a  program  in  a  single  computer,  where  there  arc  no  specific  interfaces 
between  the  CSCI  and  the  computer  (the  only  HWCl).  and  where  there  arc  no  cxicmal 
interfaces  or  i  ilcriaccs  to  other  configuration  items,  there  is  almost  certainly  no  need  for  an  IRS 
(M  IDD.  There  arc  also  projects  where  some  or  all  of  these  requirements  arc  not  met  and  the 
required  interface  information  can  be  provided  .satisfactorily  in  Ihc  SSDD  or  an  SRS.  provided 
tliat  these  documents  are  tailored  to  provide  an  adequate  level  of  detail. 

*>.4  Software  Requirements  Specification 

rhe  SRS  is  usually  the  key  contractual  specification  for  software  development,  particularly 
when  the  software  is  being  developed  by  a  subcontractor. 

In  the  authors'  experience  the  two  most  common  problems  in  SRS  preparation  arc  the  absence 
ol  a  genuine  analysis  of  the  requirements  and  the  inclusion  of  dc.sign  information  which  should 
instead  be  in  the  SDD.  Often  these  problems  occur  together  when  inexperienced  development 
stall  confuse  the  analysis  task  with  that  of  preliminary  design. 

An  SRS  should  only  contain  requirements  information  which  stems  from  a  higher  level 
statcmeni  of  requirement  (SSDD.  SSS  or  other  specification)  and  which  is  refined  as  a  result  of 
the  developer's  requirements  analysis.  Information  that  is  likely  to  change  as  a  result  of 
detailed  design  should  either  be  relegated  to  the  SDD  or  included  separately  on  an  '  infonnalion 
only  basis. 

llie  authors  have  noticed  a  reluctance  by  some  developers  to  derive  additional  requirements 
I  lorn  those  specified,  resulting  in  an  SRS  which  is  little  mote  than  a  rcficclion  of  the  higher 
level  specifications.  This  practice  is  to  the  disadvantage  of  both  the  customer  and  the  developer 
IBl'LSHl  and  should  lead  to  rejection  of  the  SRS.  The  customer  should  be  seeking  an 
indication  from  the  SRS  of  the  developer's  understanding  of  the  requirements.  Approval  of  the 
SRS  means  that  the  derived  requirements  are  then  included  in  the  allocated  baseline  and  the 
dm  elopcr  can  proceed  to  design  with  an  appnwed  .set  of  consi.sienl  and  detailed  requirements. 

Another  potential  problem  is  the  attempt  to  map  the  results  of  an  inappropriate  analysis  method 
otiio  the  structure  of  an  uniailorcd  SRS.  As  discus,scd  in  Section  3.3.  il  is  important  that  a 
developer's  analysis  and  dc.sign  methods  arc  consistent  with  the  development  framework 
described  by  2Ifi7A  If  the  requirements  of  the  SRS  and  SDD  documents  arc  ignored  it  is 
likely  that  the  documentation  ia,sk  will  be  difficult  and  the  result  will  be  of  little  value. 
Developers  should  be  wary  of  the  advice  given  by  Coad  and  Yourdon  ICOAW):  "You  i^ 
follow  good  principles  of  analysis  and  design,  and  then  figure  out  how  to  111  your  results  into 
the  21b7A  framework. "  They  would  be  better  advi.scd  to  plan  how  their  methods  will  fit  the 
h'rmats  of  the  DIDs  and  to  tailor  their  melhtxls  and  the  DIDs  accordingly. 
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The  main  lailorinjz  (or  an  SRS  is  likely  lo  he  to  accommodate  dirferent  analysis  mcihrxls.  This 
is  discussed  in  Section  7, 

9.5  Software  Design  Document 

The  SDl)  is  used  lo  document  the  design  of  a  CSC!  and  its  partitioning  into  CSCs  (including 
sub-level  CSCs  if  required)  and  CSUs,  which  relate  directly  lo  the  actual  source  code.  It  is 
unfortunate  that  the  SDD  is  often  viewed  (at  least  in  the  short  term)  only  as  a  milestone  to  be 
passed  at  CDR.  Any  tailoring  of  this  document  should  consider  its  use  during  the  development 
and  maintenance  of  the  system. 

It  can  be  infened  from  the  requirements  of  2167A  and  the  SDD  that  a  CSU  is  a  single  and 
separate  source  code  element  (see  also  Section  6.2),  containing  a  single  entry  point  and  no  data 
definitions  whose  scope  extends  outside  the  unit;  and  that  all  data  accessed  from  more  than  one 
CSU  is  global  to  the  CSCl.  No  information  is  required  on  the  scope  or  visibility  of  code  or 
data  elements. 

The  authors  believe  that  this  interpretation  of  a  CSU  and  the  SDD's  treatment  of  data  is 
incompatible  with  many  existing  design  methods  and  modem  programming  languages.  For 
example,  it  will  not  accommodate  the  adequate  documentation  of  the  following: 

•  The  dellnition  of  related  dat;i  and  subprograms  in  an  Ada  package. 

•  The  definition  of  object  classes  or  data  types  as  separate  program  elements. 

•  Elements  such  as  Ada  tasks  and  generic  packages 

In  addition,  the  level  of  description  required  ISDD  4.X.Y.21,  if  applied  to  all  subprograms,  is 
more  than  is  normally  needed  lor  development,  maintenance  or  visibility  of  the  design  by  the 
customer 

In  dte  authors'  opinion,  the  SDD  in  its  untailored  foim  is  unsuitable  for  almost  any  project.  It 
is  iherelorc  strongly  recommended  that  the  SDD  be  tailored  for  all  projects  depending  on  the 
application,  the  design  method  and  the  language  u.sed  (among  other  factors). 

The  most  difficult  issue  lacing  the  customer  and  developer  in  the  tailoring  of  the  SDD  is  not 
seen  as  the  definition  of  die  structure  of  the  document  (or  documents),  or  requirements  for  the 
documentation  of  specific  software  elements,  but  in  the  coherence  of  the  design  description  and 
the  level  of  detail  supplied.  While  the  level  of  detail  required  in  the  DoVgn  paragraph  may  be 
excessive  for  many  units,  it  will  be  necc.ssary  to  define  .some  or  all  of  this  information  in 
certain  circumstances.  Determining  firm  generic  criteria  acceptable  to  bcUh  customer  and 
developer  for  the  different  design  detail  required  for  different  CSUs  is  almost  impossible.  (It  is 
probably  this  fact  that  is  responsible  for  the  current  rigid  and  comprehensive  -  some  would  say 
draconian  -  requirements  of  the  SDD.) 

It  is  therefore  suggested  that  the  tailoring  should  include  the  form  and  content  of  the 
documentation  required  for  different  types  and  levels  of  C.SUs.  where  comprehensive  guidelines 
arc  provided  for  the  selection  of  the  level  and  hence  the  detail  required  in  their  definition.  It  is 
important  that  this  tailoring  be  agreed  prior  to  contract  signing.  Approval  or  otherwise  of  the 
choice  of  level  would  ultimately  rest  with  the  customer,  as  part  of  the  design  approval.  Tlic 
ri.sk  that  the  customer  will  not  approve  the  levels  chosen  should  be  reduced  by  informal 
discussions  between  the  customer  and  developer  prior  lo  the  commencement  of  the 
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ilocumciilation  of  detailed  design  These  may  include  written  agreements  with  regard  to  the 
content  of  the  SDD. 

The  above  approach  will  only  work  successfully  il  the  customer  has  the  capability  to  understand 
ihc  design  and  the  risks  and  advantages  in  the  different  levels  of  documentation.  Such  a 
capability  requires  appropriate  skills  and  experience,  coupled  with  regular  discussions  with  the 
developer  throughout  the  design  process  (Section  3.2). 

9.6  Software  Product  Specification  and  Version  Description  Document 

Ihe  Stdiwarc  Product  Specification  (SPS)  in  its  untailorcd  form  provides  four  items  of 
infonnation:  a  reference  to  the  relevant  SDD,  the  source  listings  or  a  reference  to  them, 
compilei/asscmblcr  identification  and  the  measumd  resource  utilisation.  It  forms  part  of  the 
product  baseline. 

I  hc  authors  see  little  value  in  the  provision  of  source  li.stings  for  modem  software  projects  if 
the  Nource  software  is  provided  in  electronic  form.  Without  source  listings,  the  SPS  is 
relatively  insubstantial  and,  unless  there  are  rea.sons  for  providing  additional  infonnation  in  the 
SPS  (by  tailoring),  il  may  be  tailored  out.  The  remaining  requirements,  if  appropriate,  may  be 
adder!  Kr  the  Vcrsioti  Description  Document  (VDD).  It  is  important,  however,  that  the  updated 
versirni  of  the  SDD  be  included  as  part  of  the  product  deliverables  (in  lieu  of  its  inclusion  in 
the  SPS).  ;ind  additional  tailoring  will  be  necessary  to  ensure  this. 

9.7  Test  documents 

It  lorinal  qualification  testing  is  rc(|uired.  the  Software  Te.st  Plan  (S'fP).  Descriptions  tS'PDs) 
and  Reports  (STRs)  wilt  be  required. 

While  the  content  and  fomiat  of  these  documents  is  adequate  in  many  cases,  Ihe  test 
descriptions  can  often  be  described  more  effectively  and  efficiently  using  a  fomiat  which  is 
specifically  developed  for  Ihc  application  being  tested  or  the  test  method  (eg  in  automated 
testing)  In  these  cases,  however,  the  principles  of  the  untailorcd  DID  should  be  followed  in 
lorrnulating  test  descriptions  that  are  unambiguous,  repeatable  and  traceable  to  the  requirements 
that  they  test.  Care  should  also  be  taken  to  avoid  excessive  duplication  which  may  lead  to 
errors  when  changes  arc  made. 

1  est  results  will  come  in  various  forms  dependent  on  the  application  and  lest  tools.  It  is 
important  that  these  be  accepted  in  their  output  fomi  (eg  printouts,  test  logs),  rather  than 
transcribing  them  into  an  unsuitable  format  in  an  STR.  The  STR  requirements  will  need  to  be 
tailored  to  accommodate  these  changes. 

9.8  Cnmputer  Resources  Integraled  Support  DocumenI 

I  he  CRISD  requirements  will  generally  be  IouikI  to  be  loo  broad  to  provide  ade(]uale 

documentation  of  the  transitioning  R-quirements.  This  DID  should  be  tailored  to  suit  specific  f 

projects  and  their  objectives  for  software  support. 

Customers  may  find  DOD-STD-1467(AR).  Software  Support  Environment,  useful  in  tailoring 
Ihe  CRISD.  Although  ptKir  in  terms  of  readability,  this  standard  provides  additional 
requirements  covering 

I 

•  the  use  of  and  compatibility  with  existing  .support  facilities, 

•  software  rights 


! 
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•  catering  lor  diircrcni  categories  of  software  Isuch  as  COTS) 

•  the  minimum  requirements  for  support  facilities,  and 

•  advice  with  regard  to  qualification  testing  of  the  support  facilities. 

9.9  Manuals  (CSOM,  SUM,  SPM  and  FSM) 

The  Computer  System  Operator's  Manual  (CSOM),  Software  User’s  Manual  (SUM),  Software 
Programmer's  Manual  (SPM)  and  Firmware  Support  Manual  (FSM)  provide  general 
requirements  which,  in  the  authors’  opinion,  arc  unlikely  to  lead  to  satisfactory  useable 
manuals.  The  fonnat  and  content  of  manuals  will  usually  be  strongly  dependent  on  the 
application  and  its  environment.  Each  of  these  DIDs,  if  required,  should  be  specifically  tailored 
for  its  intended  use. 

l>;fcncc  customers  will  often  require  their  own  standards  for  manuals,  which  will  override  the 
format  requirements  for  these  DIDs  This  should  not  necessarily  result  in  the  DIDs  being 
discarded,  however.  They  may  still  serve  a  purpose  by  specifying  the  structure,  content  and 
level  of  description  of  the  required  information. 

It  should  be  noted  diat  the  Software  Programmer’s  Manual,  despite  its  attractive  title,  is  rarely 
necessary  in  modem  software  development  projects.  It  is,  in  essence,  an  assembly  language 
manual  for  the  ctmiputer/s  used 


10  FURTHER  WORK 

t  he  authors  recognise  the  fact  that  this  study  has  examinerl  only  some  of  the  aspects  relevant  to 
software  development  using  DOD-.STD-2 167A,  and  has  concentrated  on  what,  in  their  opinion, 
are  the  most  pressing  problems.  Exposure  to  these  problems  has,  however,  identified  several 
other  areas  where  further  effort  is  likely  to  assist  in  the  management  of  21(S7A  projects,  the 
communication  between  customers  and  developers  and  the  maintainabihiy  of  software  so 
produced, 

10.1  Assessment  of  developers  and  customers 

It  has  become  evident  to  the  authors  that  some  developers  in  Australia  may  not  have  the 
knowledge,  experience  or  standards  to  develop  software  in  accordance  with  2I67A.  Similar 
deficiencies  in  customer’s  project  teams  indicate  (hat  their  ability  to  manage  software 
development  projects  is  in  question.  Guidelines  need  to  be  prepared  to  enable  customers  to 
assess  the  capability  of  potential  deveU'ixTs  and  their  own  project  staff  to  participate  in  a 
2 167 A  project. 

10.2  Preparation  ttf  RFTs  and  tenders 

The  authors’  survey  and  workshop  revealed  that  developers  tiflen  have  difficulty  in  the 
preparation  of  tenders,  in  their  understanding  of  what  is  really  w.inted  and  how  the  tenders  will 
be  interpreted  by  the  customer.  Some  arc  reluctant  to  suggest  significant  tailorings  of  2167A. 
for  example,  even  when  the  RFT  encourages  tailoring,  for  fear  that  their  lender  will  be  rejected 
on  that  basis.  Research  is  required  into  the  contents  of  RFTs  and  lenders,  to  recommend 
pmccdures  ihal  will  improve  Ihc  cost  and  quality  of  the  software  product. 
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10.3  FU'tfronic  dociiinotitalion  iiiid  conimiinirylion 

li  is  incviiiihlc  there  uill  bo  ;m  iiicroasiiig  reliance  nii  clcclrcinic  documenlaiion  and 
c(in)inunic;ili(Mi  in  IX’Icnco  [inijccls.  Wiiluiul  standardisation  this  will  result  in  major  costs  and 
disatrav,  with  the  possibility  ol  reduced  eommunieation  between  devcltrpers  and  customers  in 
the  short  temi.  I'he  Delonce  policy  to  statidardise  on  C'AI.S  should  assist  in  this  rejiard,  but 
research  is  needed  to  insestipatc  the  Ix’st  use  ol  in  software  developments  and  ttr  make 

appropriate  recommetahitions 

10.4  1  ailoriii};  for  internal  Defence  projeels 

Soltwarc  dcvelojx'd  w  ithin  Oetenco  iticludes  maintenance  of  existing  systems,  development  of 
new  systems  tm  '  prottity |X's,  iind  a  large  range  of  dilierent  types  of  .software  development 
within  nsro.  In  many  cases  sofiwtue  is  required  to  be  developet'  in  accordance  willi  2I67A. 

Inienuil  ile\ elociment  dillers  horn  dc\elo|Mncnt  by  contract  niiiinly  in  the  level  of  trust  placed  in 
the  dc\cloper  In  scune  i.ascs,  [larticularly  in  system  sup[W)rt  centres  for  existing  soltwarc.  the 
lies  eloper  m;i\  tilso  b’.'  responsihk'  hn  the  genciiititm  (d  system  level  sireci  Heat  ions  titicl  for 
osersecitig  fonnal  quahticaiton  tcstim:  In  others,  such  as  when  DS'T'O  is  responsible  for  the 
de\elo[imcnt  td  (ipcraiional  ''(diwarc.  the  customer  m;iy  have  a  more  pronounced  r(dc.  but  is 
unlikely  to  rctimre  the  level  ot  viobihiv  legardcd  ;is  necesstiry  lor  tin  exienittl  development. 

Hectiusc  ol  the  dilleicnt  requirements  loi  \i.ibilily  ol  the  design  and  review  (d  progress,  the 
l.iilonng  lor  sueh  proieci'-  will  Ix'  dilierent  Irom  those  for  extental  developments,  (luidelincs 
need  to  be  devehiped  lot  llie  t.iiloniig  ol  .Ipr’.A  in  these  circumstances. 

10.5  1)01)  S  ri>-2I67  \  resources 

I  hc  .itiihors  sec  a  need  hu  a  ei'llceiion  ol  icstHirces  in  Auslr;ili;i  assisting  cusit'incrs  and 
developers  in  the  cdtie.ilioii  and  use  ol  2I07  \.  These  shouM  inchule: 

•  ,\  compiehcn>i\ e  hbr.iiy  and  bibliography  of  relevant  iidomialitm  relating  to  2lb7,A 
pngct  Is 

•  Deliniiions  and  inicipicl.iiions  ol  2I07.A  and  DID  retjuiremenis. 

•  .Sample  Itiiloring.s  lor  dilierent  pio)ccl  ly(X's. 

•  Word  processor  lemplales  lor  2lb7A  documentation. 

10.6  (oiiriancc  for  different  applications  and  development  metlntds 

I  urtiicr  work  is  required  to  detennine  detailed  tailoring  guidelines  for  different  application  types 
and  dc  clopmcni  methods  Apidictilioiis  sh(nild  include  real-time  embcdiled  systems, 
distributed  systems,  informalitm  systems  and  safety  or  security  critical  systems.  Development 
methods  should  include  incremenitti  and  phased  developments,  object  oricnlcd  analy^'is  and 
design  techniques,  and  the  use  of  mathematically  based  ("formal  ")  methods. 

10.7  (fiiidancp  for  the  use  of  V&\'  in  2I67A  pnyeets 

Our  survey  indicated  uncertainly  in  the  use  of  Verification  and  Validation  in  Defence  projects, 
txnh  in  the  application  of  VAtV  in  the  development  process  (internal  or  independent)  and  in  the 
monitoring  of  the  V&V  activities  by  Ihc  customer,  f-urther  work  is  required  to  investigate 
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V&V  si;indariis,  ttic  value  o(  independent  V&V,  and  suitable  frameworks  for  interfaces  between 
the  various  V&V  staff  and  the  development  team. 

10.8  Follow-up  studies 

Although  DOD-S'rD-2167A  is  a  development  standard,  it  will  have  effects  on  the  support  of 
software  for  many  years  after  development.  As  yet  these  effects  cannot  be  confidently  assessed. 
T  he  results  and  recommendations  of  this  study  should  be  reviewed  in  two  to  three  years  time  in 
the  light  of  further  experiences,  particularly  with  regard  to  the  support  of  software  developed  to 
the  rei]uircmcnts  of  2167A. 

lO.*)  (ipdate  for  DOD-STD-2I67B 

The  I'S  f>oD  software  standards  community  is  developing  the  next  version  of  DOD-STD-2167 
under  the  working  title  of  MIL-STD-SDD  (for  Software  Development  and  Documentation).  It 
is  expected  that  this  standard  will  combine  (harmonise)  the  requirements  of  DOD-STD-2I67A 
and  DOD-STD-7935.  Apart  from  any  contributions  which  may  be  made  to  tiiis  endeavour, 
such  as  the  results  of  this  study,  there  is  likely  to  be  a  need  to  analyse  the  new  standard  for  use 
in  .Australian  stdTware  developments. 


11  CONCLUSIONS 

The  course  of  D017-STD-2167A  soltware  development  projects  in  Australia  has  not  generally 
been  smooth,  but  neither  was  the  course  of  such  projects  prior  to  the  introduction  of  the 
standard.  There  tire,  however,  specific  problems  which  have  been  highlighted  by  the  use  of 
2167A, 

I'hc  prime  problem,  which  is  a  major  contributor  to  most  of  the  other  difficulties,  is  the  lack  of 
experience  and  understanding  of  customers  and  developers  both  in  the  use  of  2167A  and  in 
software  development  management.  This  deficiency  has  led  to  many  other  problems  including 
in;ide(|i);iie  tmd  inappropriate  ttiiloring  of  the  standard,  poor  or  even  unusable  documentation, 
cost  ami  schedule  overruns,  and  software  products  of  low  quality.  Another  serious  problem  is 
the  lack  of  communication  between  customers  and  developers  in  development  projects. 

It  is  hoped  that  the  recommendations  and  advice  in  this  paper  will  help  to  improve  this 
situation,  in  making  customers  and  developers  aware  of  the  difficullics  they  face,  and  providing 
some  guidance  in  their  avoidance  or  resolution. 

The  most  important  recommendation,  however,  is  that  both  customers  and  developers  must 
dedicate  more  effon  to  the  education  of  their  staff,  so  that  they  arc  more  able  to  cope  with  the 
development  of  complex  software  systems  to  a  standard  which  docs  not  forgive  ignorance  or 
amateurism. 
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